Page MenuHomePhabricator
Create Task
Maniphest T431978

Enable FP8 KV Cache for Qwen36-27b
Closed, ResolvedPublic
Actions

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
ml-services: bump llm-qwen36-27b image and set fp8 KV cache dtypeBartosz Wójtowiczoperations/deployment-chartsmaster+3 -1
qwen36: Allow configuring vLLM KV cache dtypeBartosz Wójtowiczmachinelearning/liftwing/inference-servicesmain+39 -0
Customize query in gerrit

Related Objects
Search...

Event Timeline

BWojtowicz-WMF created this task.Mon, Jul 13, 8:56 AM
gerritbot added a comment.Mon, Jul 13, 8:57 AM

Change #1309662 had a related patch set uploaded (by Bartosz Wójtowicz; author: Bartosz Wójtowicz):

[machinelearning/liftwing/inference-services@main] qwen36: Allow configuring vLLM KV cache dtype

https://gerrit.wikimedia.org/r/1309662

gerritbot added a project: Patch-For-Review.Mon, Jul 13, 8:57 AM
gerritbot added a comment.Mon, Jul 13, 9:06 AM

Change #1309662 merged by jenkins-bot:

[machinelearning/liftwing/inference-services@main] qwen36: Allow configuring vLLM KV cache dtype

https://gerrit.wikimedia.org/r/1309662

jenkins-bot mentioned this in rMLIS6a27465f8aa6: qwen36: Allow configuring vLLM KV cache dtype.Mon, Jul 13, 9:06 AM
Maintenance_bot removed a project: Patch-For-Review.Mon, Jul 13, 9:31 AM
gerritbot added a comment.Mon, Jul 13, 1:07 PM

Change #1310091 had a related patch set uploaded (by Bartosz Wójtowicz; author: Bartosz Wójtowicz):

[operations/deployment-charts@master] ml-services: bump llm-qwen36-27b image and set fp8 KV cache dtype

https://gerrit.wikimedia.org/r/1310091

gerritbot added a project: Patch-For-Review.Mon, Jul 13, 1:07 PM
gerritbot added a comment.Mon, Jul 13, 1:39 PM

Change #1310091 merged by jenkins-bot:

[operations/deployment-charts@master] ml-services: bump llm-qwen36-27b image and set fp8 KV cache dtype

https://gerrit.wikimedia.org/r/1310091

Maintenance_bot removed a project: Patch-For-Review.Mon, Jul 13, 2:32 PM
achou moved this task from Unsorted/Needs Triage to In Progress on the Machine-Learning-Team (Q1 FY2026-27) board.Tue, Jul 14, 11:10 AM
achou mentioned this in T431846: Q1 FY2026-27 Milestone 1: Deliver a V0 LLM platform for Wikimania.Fri, Jul 17, 2:41 PM
BWojtowicz-WMF closed this task as Resolved.Tue, Jul 21, 12:15 PM
BWojtowicz-WMF moved this task from In Progress to Done on the Machine-Learning-Team (Q1 FY2026-27) board.
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits