Page MenuHomePhabricator
Create Task
Maniphest T293331

Migrate from Kfserving to Kserve
Closed, ResolvedPublic
Actions

Description

The first Kserve release is out: https://github.com/kserve/kserve/releases/tag/v0.7.0

I am strongly in favor of migrating to it before the release of our MVP, since we more easily upgrade at this stage.

Overall things to do:

  1. create new kserve docker images in the production-images repo (controller, storage-initializer, etc..)
  2. add a new helm chart for kserve, getting the yaml files from upstream (and apply all the custom changes that we do for kfserving)
  3. Upgrade inference-services' docker image to use kserve 0.7
  4. Deploy everything :)

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
topic: migrate from kfserving to kserveAccrazemachinelearning/liftwing/inference-servicesmain+92 -55
Update docs to reflect KServeKevin Baziramachinelearning/liftwing/inference-servicesmain+29 -29
ml-services: update revscoring-articlequality imgAccrazeoperations/deployment-chartsmaster+1 -1
ml-services: update revscoring-draftquality imageAccrazeoperations/deployment-chartsmaster+2 -2
ml-services: update revscoring-editquality imageAccrazeoperations/deployment-chartsmaster+1 -1
outlink: migrate transformer to kserveAccrazemachinelearning/liftwing/inference-servicesmain+101 -8
outlink: migrate from kfserving to kserveAccrazemachinelearning/liftwing/inference-servicesmain+98 -8
draftquality: migrate from kfserving to kserveAccrazemachinelearning/liftwing/inference-servicesmain+82 -46
articlequality: fix kserve importsAccrazemachinelearning/liftwing/inference-servicesmain+13 -13
editquality: migrate from kfserving to kserveAccrazemachinelearning/liftwing/inference-servicesmain+85 -43
articlequality: upgrade transformer to kserveAccrazemachinelearning/liftwing/inference-servicesmain+1 -1
articlequality: migrate kfserving dep to kserveAccrazemachinelearning/liftwing/inference-servicesmain+81 -43
Remove any trace of Kubeflow KfservingElukeyoperations/deployment-chartsmaster+23 -17 K
Remove Kubeflow KfservingElukeyoperations/docker-images/production-imagesmaster+0 -151
helmfile.d: update the deploy-kserve cluster role to KServeElukeyoperations/deployment-chartsmaster+1 -1
Move the ml-serve cluster to KServeElukeyoperations/deployment-chartsmaster+45 -9
Add the kserve and kserve-inference chartsElukeyoperations/deployment-chartsmaster+13 K -0
Add the KServe imagesElukeyoperations/docker-images/production-imagesmaster+86 -0
Show related patches Customize query in gerrit

Related Objects
Search...

StatusSubtypeAssignedTask
 ResolvedNoneT272917 Lift Wing proof of concept
 Resolved• ACrazeT293331 Migrate from Kfserving to Kserve

Event Timeline

elukey created this task.Oct 14 2021, 8:19 AM
gerritbot added a comment.Oct 14 2021, 10:11 AM

Change 730722 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/docker-images/production-images@master] Add the KServe images

https://gerrit.wikimedia.org/r/730722

gerritbot added a project: Patch-For-Review.Oct 14 2021, 10:11 AM
gerritbot added a comment.Oct 14 2021, 11:16 AM

Change 730761 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] Add the kserve and kserve-inference charts

https://gerrit.wikimedia.org/r/730761

gerritbot added a comment.Oct 14 2021, 11:16 AM

Change 730762 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] Move the ml-serve cluster to KServe

https://gerrit.wikimedia.org/r/730762

gerritbot added a comment.Oct 14 2021, 12:39 PM

Change 730722 merged by Elukey:

[operations/docker-images/production-images@master] Add the KServe images

https://gerrit.wikimedia.org/r/730722

gerritbot added a comment.Oct 14 2021, 1:14 PM

Change 730761 merged by Elukey:

[operations/deployment-charts@master] Add the kserve and kserve-inference charts

https://gerrit.wikimedia.org/r/730761

gerritbot added a comment.Oct 14 2021, 1:48 PM

Change 730762 merged by Elukey:

[operations/deployment-charts@master] Move the ml-serve cluster to KServe

https://gerrit.wikimedia.org/r/730762

gerritbot added a comment.Oct 14 2021, 1:58 PM

Change 730807 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] helmfile.d: update the deploy-kserve cluster role to KServe

https://gerrit.wikimedia.org/r/730807

gerritbot added a comment.Oct 14 2021, 2:03 PM

Change 730807 merged by Elukey:

[operations/deployment-charts@master] helmfile.d: update the deploy-kserve cluster role to KServe

https://gerrit.wikimedia.org/r/730807

gerritbot added a comment.Oct 14 2021, 3:12 PM

Change 730831 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/deployment-charts@master] Remove any trace of Kubeflow Kfserving

https://gerrit.wikimedia.org/r/730831

gerritbot added a comment.Oct 14 2021, 3:13 PM

Change 730832 had a related patch set uploaded (by Elukey; author: Elukey):

[operations/docker-images/production-images@master] Remove Kubeflow Kfserving

https://gerrit.wikimedia.org/r/730832

gerritbot added a comment.Oct 14 2021, 4:32 PM

Change 730832 merged by Elukey:

[operations/docker-images/production-images@master] Remove Kubeflow Kfserving

https://gerrit.wikimedia.org/r/730832

gerritbot added a comment.Oct 14 2021, 4:39 PM

Change 730831 merged by Elukey:

[operations/deployment-charts@master] Remove any trace of Kubeflow Kfserving

https://gerrit.wikimedia.org/r/730831

• ACraze mentioned this in T293677: Sunset MiniKF sandboxes.Oct 18 2021, 6:16 PM
• ACraze added a comment.Nov 29 2021, 5:13 PM

From IRC (wikimedia-ml) 2021-11-22:

elukey: folks I think that we should migrate to KServer from kserve==0.7, I suspect that https://github.com/kserve/kserve/commit/c10e6271897d7fd058f5618d5e0e70b31496f64c is very important

Also @elukey flagged an issue in T296173#7520325 where our older versions of kfserving python library were causing strange results during initial load testing.

I think we should prioritize upgrading the inference services to kserve 0.7, I think we are in a good place with the new ml-sandbox to start attempting to build/test upgraded isvcs starting this week.

gerritbot added a comment.Dec 3 2021, 5:18 PM

Change 734432 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] articlequality: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/734432

gerritbot added a comment.Dec 3 2021, 7:16 PM

Change 743487 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] articlequality: upgrade transformer to kserve

https://gerrit.wikimedia.org/r/743487

gerritbot added a comment.Dec 3 2021, 7:54 PM

Change 734432 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] articlequality: migrate kfserving dep to kserve

https://gerrit.wikimedia.org/r/734432

gerritbot added a comment.Dec 3 2021, 8:40 PM

Change 743496 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] editquality: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/743496

gerritbot added a comment.Dec 6 2021, 3:46 PM

Change 734432 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] articlequality: migrate kfserving dep to kserve

https://gerrit.wikimedia.org/r/734432

gerritbot added a comment.Dec 6 2021, 3:46 PM

Change 743487 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] articlequality: upgrade transformer to kserve

https://gerrit.wikimedia.org/r/743487

• ACraze mentioned this in rMLIS5305dc3332df: articlequality: upgrade transformer to kserve.Dec 6 2021, 3:49 PM
• ACraze mentioned this in rMLISdb42d38cc74f: articlequality: migrate kfserving dep to kserve.
gerritbot added a comment.Dec 8 2021, 4:26 PM

Change 745281 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] articlequality: fix kserve imports

https://gerrit.wikimedia.org/r/745281

gerritbot added a comment.Dec 9 2021, 5:39 AM

Change 743496 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] editquality: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/743496

gerritbot added a comment.Dec 9 2021, 5:39 AM

Change 745281 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] articlequality: fix kserve imports

https://gerrit.wikimedia.org/r/745281

• ACraze mentioned this in rMLISb73d602b445f: articlequality: fix kserve imports.Dec 9 2021, 5:40 AM
• ACraze mentioned this in rMLISc8dd6e71bc66: editquality: migrate from kfserving to kserve.
gerritbot added a comment.Dec 9 2021, 5:34 PM

Change 745571 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] draftquality: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745571

gerritbot added a comment.Dec 9 2021, 7:14 PM

Change 745583 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] topic: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745583

gerritbot added a comment.Dec 10 2021, 6:33 AM

Change 745571 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] draftquality: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745571

gerritbot added a comment.Dec 10 2021, 6:33 AM

Change 745583 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] topic: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745583

• ACraze mentioned this in rMLIS92b454914f54: topic: migrate from kfserving to kserve.Dec 10 2021, 6:33 AM
• ACraze mentioned this in rMLISf3b1c8ebd5f8: draftquality: migrate from kfserving to kserve.
gerritbot added a comment.Dec 10 2021, 5:44 PM

Change 745922 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] outlink: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745922

gerritbot added a comment.Dec 10 2021, 6:13 PM

Change 745925 had a related patch set uploaded (by Accraze; author: Accraze):

[machinelearning/liftwing/inference-services@main] outlink: migrate transformer to kserve

https://gerrit.wikimedia.org/r/745925

gerritbot added a comment.Dec 13 2021, 7:01 AM

Change 745922 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] outlink: migrate from kfserving to kserve

https://gerrit.wikimedia.org/r/745922

gerritbot added a comment.Dec 13 2021, 7:01 AM

Change 745925 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] outlink: migrate transformer to kserve

https://gerrit.wikimedia.org/r/745925

• ACraze mentioned this in rMLISe81491932c3a: outlink: migrate transformer to kserve.Dec 13 2021, 7:09 AM
• ACraze mentioned this in rMLIS3a433a8673a4: outlink: migrate from kfserving to kserve.
gerritbot added a comment.Dec 13 2021, 5:03 PM

Change 746898 had a related patch set uploaded (by Accraze; author: Accraze):

[operations/deployment-charts@master] ml-services: update revscoring-editquality image

https://gerrit.wikimedia.org/r/746898

gerritbot added a comment.Dec 13 2021, 5:16 PM

Change 746901 had a related patch set uploaded (by Accraze; author: Accraze):

[operations/deployment-charts@master] ml-services: update revscoring-draftquality image

https://gerrit.wikimedia.org/r/746901

gerritbot added a comment.Dec 13 2021, 5:28 PM

Change 746898 merged by Elukey:

[operations/deployment-charts@master] ml-services: update revscoring-editquality image

https://gerrit.wikimedia.org/r/746898

gerritbot added a comment.Dec 13 2021, 5:28 PM

Change 746901 merged by Elukey:

[operations/deployment-charts@master] ml-services: update revscoring-draftquality image

https://gerrit.wikimedia.org/r/746901

gerritbot added a comment.Dec 13 2021, 8:11 PM

Change 746949 had a related patch set uploaded (by Accraze; author: Accraze):

[operations/deployment-charts@master] ml-services: update revscoring-articlequality img

https://gerrit.wikimedia.org/r/746949

gerritbot added a comment.Dec 14 2021, 7:20 AM

Change 746949 merged by Elukey:

[operations/deployment-charts@master] ml-services: update revscoring-articlequality img

https://gerrit.wikimedia.org/r/746949

• ACraze closed this task as Resolved.Dec 14 2021, 5:08 PM
• ACraze claimed this task.

All inference-service images in the WMF registry have been upgraded to kserve v0.7.0, the ml-services helmcharts are updated and have been deployed to the ml-serve eqiad cluster.

Going to mark this as RESOLVED for now. Feel free to reopen if there are any tasks I missed.

gerritbot added a comment.Jan 10 2022, 9:27 AM

Change 752612 had a related patch set uploaded (by Kevin Bazira; author: Kevin Bazira):

[machinelearning/liftwing/inference-services@main] Update docs to reflect KServe

https://gerrit.wikimedia.org/r/752612

gerritbot added a comment.Jan 10 2022, 4:58 PM

Change 752612 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] Update docs to reflect KServe

https://gerrit.wikimedia.org/r/752612

kevinbazira mentioned this in rMLISc1eea21d8be8: Update docs to reflect KServe.Jan 10 2022, 4:58 PM
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits