| DPogorzelski-WMF | |
| Sep 11 2026, 1:08 PM |
Build and deploy a working instance of liftwing studio on dse eqiad cluster.
This involves the deployment of a pgsql instance (done) and the creation of a dedicated liftwing studio chart.
This task is to track the progress on the chart work.
Liftwing studio is a web frontend to the LLM models hosted by the ML team on ml-serve k8s clusters.
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | DPogorzelski-WMF | T439140 LiftWing Studio | |||
| Resolved | DPogorzelski-WMF | T437706 Liftwing studio chart |
Change #1339756 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: chart for Open WebUI and LiteLLM
There is a candidate chart that i will test out of band before submitting. The pgsql isntance is up and running and secrets in the private repo are set up and ready.
The chart is working i will be investigating if/how it connects to the llm models on ML cluster.
I will also add 2 small changes to expose the service on a domain and make sure it's routed correctly.
After that i will investigate central authentication.
The service will not be exposed to the outside world until this is done.
Change #1341128 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/dns@master] dns: add liftwing-studio records
Change #1341130 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/puppet@production] liftwing-studio: add to service catalog
Change #1341163 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/puppet@production] services_proxy: send SNI on the inference listener
Change #1341174 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: chart deployment
Change #1341130 merged by Dpogorzelski:
[operations/puppet@production] liftwing-studio: add to service catalog
Change #1341163 merged by Dpogorzelski:
[operations/puppet@production] services_proxy: send SNI on the inference listener
Change #1339756 merged by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: chart for Open WebUI and LiteLLM
Change #1341128 merged by Dpogorzelski:
[operations/dns@master] dns: add liftwing-studio records
Change #1341174 merged by jenkins-bot:
[operations/deployment-charts@master] liftwing-studio: chart deployment
Change #1341903 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: add external-services egress to the chart
Change #1341904 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: OIDC auth against idp.wikimedia.org
Change #1341909 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: OIDC auth against idp.wikimedia.org
Change #1341909 abandoned by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: OIDC auth against idp.wikimedia.org
Reason:
Duplicate: created accidentally when an amend lost the Change-Id. Superseded by patchset 2 of change 1341904.
Change #1342135 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/puppet@production] idp: add the liftwing_studio service
Change #1341903 merged by jenkins-bot:
[operations/deployment-charts@master] liftwing-studio: add external-services egress to the chart
Change #1341904 merged by jenkins-bot:
[operations/deployment-charts@master] liftwing-studio: OIDC auth against idp.wikimedia.org
Change #1342135 merged by Dpogorzelski:
[operations/puppet@production] idp: add the liftwing_studio service
SSO login works and it's the only form of authentication allowed now.
Only wmf and nda groups are allowed. All new users are regular users without admin permissions.
I will now switch the setup to contact inference models via sidecar proxy.
Change #1342606 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] mesh: Accept websocket upgrades in configuration 1.15.4
Change #1342607 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: accept websocket upgrades at the tls terminator
Change #1342611 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] Copy mesh.configuration 1.15.3 -> 1.15.4
Change #1342612 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] mesh.configuration: Accept websocket upgrades
Change #1342613 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: accept websocket upgrades at the tls terminator
Change #1342606 abandoned by Dpogorzelski:
[operations/deployment-charts@master] mesh: Accept websocket upgrades in configuration 1.15.4
Reason:
wrong structure
Change #1342607 abandoned by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: accept websocket upgrades at the tls terminator
Reason:
wrong structure
There is a proposed patch to fix websocket connections, waiting for approval as there is another similar solution pending.
I will also start looking into swapping openweb ui with libre chat as this is what the ML team is leaning towards as the final choice.
For tracking purposes this is the change meant to handle websockets: https://gerrit.wikimedia.org/r/c/operations/deployment-charts/+/1338751
Change #1345056 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] ferretdb: make the Service name configurable
Change #1345057 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: back the database with ferretdb
Change #1345058 had a related patch set uploaded (by Dpogorzelski; author: Dpogorzelski):
[operations/deployment-charts@master] liftwing-studio: replace Open WebUI with LibreChat
Change #1345056 merged by Dpogorzelski:
[operations/deployment-charts@master] ferretdb: make the Service name configurable
Change #1345057 merged by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: back the database with ferretdb
Change #1345058 merged by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: replace Open WebUI with LibreChat
I swapped out openwebui with libre chat, currently can be tested here: https://liftwing-studio.discovery.wmnet:30443
I will close this task now and open a new one to review and publish this service under a public URL.
Change #1342613 abandoned by Dpogorzelski:
[operations/deployment-charts@master] liftwing-studio: accept websocket upgrades at the tls terminator
Reason:
we switched to librechat so this is not needed anymore
Change #1342612 abandoned by Dpogorzelski:
[operations/deployment-charts@master] mesh.configuration: Accept websocket upgrades
Reason:
we switched to librechat so this is not needed anymore
Change #1342611 abandoned by Dpogorzelski:
[operations/deployment-charts@master] mesh.configuration: Copy 1.15.3 -> 1.15.4
Reason:
we switched to librechat so this is not needed anymore