Page MenuHomePhabricator

Lift Wing model registry
Open, Needs TriagePublic

Description

As an engineer/data scientist/Lift Wing user,

I would like to have a model registry which I can use as the source of truth for the deployed models and their versions. offering two main benefits:

  • Versioning and tracking of models: this would allow an easier access to model version and tracking in a git like fashion
  • Collaboration and reproducibility: in order to download a model the user only needs to interact with the registry.

An potential proposal would be a git repository with DVC that does the following:

  • Holds information about the model (e.g. version) along with its s3 location. By using DVC we can easily checkout/download the model
  • ideally we can also store the model cards along with the metadata. (or the model card's URL)

Notes/Considerations:

  • UI: although a UI is not necessary it makes the model registry more powerful - especially in the case where it is bundled with the model's API documentation (Swagger UI).
  • We would have to figure out a (nice) way to integrate this with the deployment charts repo in order to get the model version we need to deploy. But this is not necessary (and can also be a controversial topic for the GitOps world) and we can consider declaring model version in both places