Qwen3-14b was successfully running on 2x192GB partitions on ml-serve1012 host.
We tried moving the host to ml-serve1013 to use 2x24GB partitions to host the model: patch1, patch2.
The deployment was successful and the model server started. However, the output from the model was corrupted - it returned the same token multiple times, which suggests there are numerical present in the model.
Example:
~ curl -s --http1.1 "https://inference.svc.eqiad.wmnet:30443/openai/v1/chat/completions" -H "Host: llm-qwen3-14b-predictor.llm.wikimedia.org" -H "Content-Type: application/json" -d '{"model":"llm-qwen3-14b","messages":[{"role":"user","content":"What is 2+2?"}],"max_tokens":50}'
{"id":"990516f0c6255553","object":"chat.completion","created":1783332682,"model":"llm-qwen3-14b","choices":[{"index":0,"message":{"role":"assistant","content":"!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!","refusal":null,"annotations":null,"audio":null,"function_call":null,"tool_calls":[],"reasoning":null,"reasoning_content":null},"logprobs":null,"finish_reason":"length","stop_reason":null,"token_ids":null}],"service_tier":null,"system_fingerprint":null,"usage":{"prompt_tokens":19,"total_tokens":69,"completion_tokens":50,"prompt_tokens_details":null},"prompt_logprobs":null,"prompt_token_ids":null,"kv_transfer_params":null}