Explore options to run TTS models for evaluation
Closed, ResolvedPublic
Actions

Assigned To

Authored By

	kevinbazira
	Apr 24 2026, 6:29 PM

Project Tags

Referenced Files

	F91205751: TTS prototype UI: search and filter pre-generated audio in library.png
	Jun 30 2026, 5:51 AM

	F91205393: TTS prototype UI: audio generation, listening, and follow-along highlighting.png
	Jun 30 2026, 5:51 AM

	F79759624: User interacting with the TTS prototype - 2026-05-05 09-16-19.mp4
	May 5 2026, 7:46 AM

	F78481394: tts_test_vibevoice_generated.wav
	Apr 28 2026, 11:18 AM

	F78481308: tts_test_kokoro.wav
	Apr 28 2026, 11:18 AM

	F78344234: tts_test_bark-small.wav
	Apr 27 2026, 2:57 PM

	F78344206: tts_test_speecht5_tts.wav
	Apr 27 2026, 2:57 PM

	F78344137: tts_test_mms-tts-eng.wav
	Apr 27 2026, 2:57 PM

Subscribers

Description

In T419288, the Readers teams have requested the ML team to host a Text-to-Speech (TTS) model on LiftWing.

At this stage, no model has been selected yet. The Readers teams would like a practical way that would enable them to test a small number of candidate HuggingFace (HF) TTS models locally first, including different voices and languages, so they can choose the most suitable open-source model before we proceed with production hosting on LiftWing.

Below are the options we are going to explore and recommend the most feasible one to the Readers teams' Software Engineers:

Use an HF TTS model that is supported by HF Inference API
Run an HF TTS model locally (using HF transformers or vLLM)
Deploy an HF TTS model on LiftWing in the experimental ns (using vLLM and KServe) or Toolforge

Related Objects
Search...

Status	Assigned	Task
Resolved	kevinbazira	T419288 Q4 FY2025-26 Goal: Text-to-Speech V0 Prototype
Resolved	kevinbazira	T424378 Explore options to run TTS models for evaluation
Resolved	taavi	T425804 Request increased quota for wiki-tts Toolforge tool
Declined	None	T425909 Request creation of wikitts VPS project
Resolved	kevinbazira	T426756 Fix text normalization edge cases in TTS prototype
Resolved	kevinbazira	T427173 Add section heading announcements in TTS prototype
Resolved	kevinbazira	T427262 Add static audio URL for native OS audio players in TTS prototype
Resolved	kevinbazira	T427488 Add word-level timestamps in TTS prototype
Resolved	kevinbazira	T428435 TTS batch generation pipeline for Featured Articles

Event Timeline

kevinbazira created this task.Apr 24 2026, 6:29 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptApr 24 2026, 6:29 PM

kevinbazira moved this task from Unsorted/Needs Triage to In Progress on the Machine-Learning-Team (Q1 FY2026-27) board.Apr 24 2026, 6:34 PM

I have checked HF for TTS models supported by the free HF Inference API and found none:

1.Using the Hub's built-in filters in the URL:
https://huggingface.co/models?pipeline_tag=text-to-speech&inference_provider=hf-inference&sort=trending

2.Programmatically via Python:

>>> from huggingface_hub import HfApi
>>> 
>>> api = HfApi()
>>> 
>>> models = api.list_models(
...     pipeline_tag="text-to-speech",
...     inference_provider="hf-inference"
... )
>>> 
>>> supported_models =[]
>>> for model in models:
...     print(f"Model ID: {model.id}")
...     supported_models.append(model.id)
... 
>>> 
>>> supported_models
[]

I was looking through the TTS candidates Ilias listed in T419288#11803859 and Kokoro (82M) really caught my eye. I played around with their HF demo and was fascinated that the audio quality is great for only 82M parameters.

Input text:
"Wikipedia is a free online encyclopedia written and maintained by a community of volunteers, known as Wikipedians, through open collaboration and the wiki software MediaWiki. Founded by Jimmy Wales and Larry Sanger in 2001, Wikipedia has been hosted since 2003 by the Wikimedia Foundation, an American nonprofit organization funded mainly by donations from readers. Wikipedia is the largest and most-read reference work in history."

Generated audio:

It took about 9s to generate this audio on CPU (the demo allows you to choose GPU or CPU).

I was curious why it worked so well, and found this post from the authors explaining their approach to G2P (Grapheme-to-Phoneme):

Instead of relying on implicit G2P, larger speech models implicitly learn this task by eating many thousands of hours of audio data. They often use a 500M+ parameter LLM at the front to predict latent audio tokens over a learned codebook, then decode these tokens into audio ... Kokoro instead relies on G2P preprocessing... and thus needs less audio to learn. We can cherrypick high-fidelity audio... which helps explain why Kokoro is very competitive.

The inference code in their demo looks pretty straightforward, and they provide an inference library for this model, which is a simple pip install.
Might be a low-hanging fruit candidate for us to try out locally or Lift Wing experimental ns. Wdyt?

Thank you for sharing your exploration of the Kokoro HF demo @achou, the Kokoro model is indeed fascinating. Since there were no freely hosted HF TTS models found in T424378#11859241, I explored the option of paid HF inference providers:
https://huggingface.co/models?pipeline_tag=text-to-speech&inference_provider=all&sort=trending

This currently shows 4 models, all hosted by fal-ai:

I tested the Kokoro model hosted by fal-ai using the HF Inference API:

import os
from huggingface_hub import InferenceClient

client = InferenceClient(
    provider="fal-ai",
    api_key="<wmf_hf_token_redacted>",
)

audio_bytes = client.text_to_speech(
    "Hello Wikipedia TTS!",
    model="hexgrad/Kokoro-82M",
)

output_filename = "tts_test_kokoro.wav"
with open(output_filename, "wb") as f:
    f.write(audio_bytes)

Below is the generated audio:

If the credits available in the HF Wikimedia paid organization account are sufficient, then the Readers Teams' Software Engineers could use a WMF HF token to test the HF TTS models.

In T424378#11860001, we found that there is a limited selection of 4 models available that can be tested using the HF Inference API and WMF HF token.

For the Readers Teams to be able to evaluate a wider variety of TTS models beyond these 4, I have looked into the option of running HF TTS models locally using HF transformers.

I started with smaller (and generally older) TTS models, as they require low resources and are more accessible for local testing. Below are the results:

1.facebook/mms-tts-eng

import soundfile as sf
from transformers import pipeline

synthesiser = pipeline("text-to-speech", model="facebook/mms-tts-eng")
output = synthesiser("Hello Wikipedia TTS!")
audio_data = output["audio"][0] if len(output["audio"].shape) > 1 else output["audio"]
sample_rate = output["sampling_rate"]
sf.write("tts_test_mms-tts-eng.wav", audio_data, samplerate=sample_rate)

Resources used: ~139MB disk space, ~1.4GB RAM
Generated tts_test_mms-tts-eng.wav:

2.microsoft/speecht5_tts

import soundfile as sf
import torch
from transformers import pipeline
from datasets import load_dataset

synthesiser = pipeline("text-to-speech", model="microsoft/speecht5_tts")
forward_params = {}
embeddings_dataset = load_dataset("regisss/cmu-arctic-xvectors", split="validation")
speaker_embedding = torch.tensor(embeddings_dataset[7306]["xvector"]).unsqueeze(0)
forward_params["speaker_embeddings"] = speaker_embedding
output = synthesiser("Hello Wikipedia TTS!", forward_params=forward_params)
audio_data = output["audio"][0] if len(output["audio"].shape) > 1 else output["audio"]
sample_rate = output["sampling_rate"]
sf.write("tts_test_speecht5_tts.wav", audio_data, samplerate=sample_rate)

Resources used: ~1.1G disk space, ~986.17MB RAM
Generated tts_test_speecht5_tts.wav:

3.suno/bark-small

import soundfile as sf
from transformers import pipeline

synthesiser = pipeline("text-to-speech", model="suno/bark-small")
forward_params = {"do_sample": True}
output = synthesiser("Hello Wikipedia TTS!", forward_params=forward_params)
audio_data = output["audio"][0] if len(output["audio"].shape) > 1 else output["audio"]
sample_rate = output["sampling_rate"]
sf.write("tts_test_bark-small.wav", audio_data, samplerate=sample_rate)

Resources used: ~3.2G disk space, ~1.8 GB RAM
Generated tts_test_bark-small.wav:

These smaller models can run locally with relatively low resource requirements, making them accessible for most development environments. However, audio quality varies: both tts_test_mms-tts-eng.wav and tts_test_speecht5_tts.wav have issues in pronunciation (i.e "Wikipedia TTS" not clearly articulated) and are not fully comprehensible; tts_test_bark-small.wav is more comprehensible, but its tone may need tuning. If the Readers Teams' Software Engineers can run these smaller HF TTS models locally, they will be able to evaluate a variety and choose one that meets their requirements.

In T424378#11861629, we found that while smaller TTS models can run locally with low resource requirements, their audio output is sometimes of low quality and not always fully comprehensible.

To address this, I explored running more recent HF TTS models locally that rank well on the HF TTS leaderboard and artificialanalysis TTS leaderboard. Below are the results:

1.hexgrad/Kokoro-82M

from kokoro import KPipeline
import soundfile as sf

# 'a' = American English, 'b' = British English
pipeline = KPipeline(lang_code='a', device='cpu')
generator = pipeline("Hello Wikipedia TTS!", voice='af_heart', speed=1.0)
for i, (graphemes, phonemes, audio) in enumerate(generator):
    sf.write("tts_test_kokoro.wav", audio, samplerate=24000)

Resources used: ~313MB disk space, ~732.59MB RAM
Generated tts_test_kokoro.wav:

2.microsoft/VibeVoice-Realtime-0.5B

$ git clone https://github.com/microsoft/VibeVoice.git
$ cd VibeVoice
$ pip install .
$ python3 demo/realtime_model_inference_from_file.py \
--model_path microsoft/VibeVoice-Realtime-0.5B \
--txt_path tts_test_vibevoice.txt \
--speaker_name Carter \
--cfg_scale 2.5 \
--output_dir ./outputs

Resources used: ~1.9GB disk space, ~4GB RAM
Generated tts_test_vibevoice_generated.wav:

These newer models still run locally with manageable resource requirements. Their audio quality is much better, speech is more natural, and comprehensible. Some of these models (e.g VibeVoice) support multiple voices, which is useful for evaluation across different tones and styles. If the Readers Teams' Software Engineers can run these more recent HF TTS models locally, they will be able to evaluate a variety and choose one that meets their requirements.

NOTE: I tried running the fishaudio/s2-pro that is currently leading the artificialanalysis TTS leaderboard, but the docs recommend using a GPU with ~24GB VRAM, which is not feasible for a local dev environment.

kevinbazira mentioned this in T419288: Q4 FY2025-26 Goal: Text-to-Speech V0 Prototype.Apr 28 2026, 12:24 PM

SherryYang-WMF subscribed.Apr 28 2026, 7:28 PM

JTannerWMF subscribed.Apr 28 2026, 8:22 PM

As Dmitry specified on slack, the Apps team's key requirement for this initial phase is an API endpoint that serves .mp3 URLs based on Wikipedia article sections.

To support this, I set up a TTS prototype using Redis, Celery, FastAPI, and the Kokoro-82M model running on CPU. Below are steps I've used to run this prototype on the ml-lab machine to fetch Wikipedia articles, clean the text (remove citation brackets, etc), split content by section, and generate .mp3 files asynchronously.

1.Run container

Spin up a python container:

$ docker run --network=host -it \
--user root \
--entrypoint /bin/bash \
python:3.11

$ cd srv

2.Install dependencies

Install system-level dependencies for audio processing (ffmpeg) and the message broker (redis-server):

$ apt-get update && apt-get install -y git curl vim ffmpeg redis-server

Install python dependencies:

$ pip install kokoro soundfile wikipedia-api numpy celery redis fastapi uvicorn

3.Download the HF TTS model

Download the Kokoro model

$ hf download hexgrad/Kokoro-82M

Confirm Kokoro model has been downloaded

$ ls ~/.cache/huggingface/hub/

4.Start Redis

Start the service:

$ service redis-server start

Confirm service status:

$ service redis-server status

Confirm redis db is responding to requests

$ redis-cli ping

5.Start Celery

Before proceeding, make sure that: Kokoro model is downloaded from HF, redis-server is running in the background, and tts_prototype code (main.py, worker.py, and index.html) is present.

In terminal 1, start the celery worker. Run this in the same directory as worker.py

$ celery -A worker worker --loglevel=info

6.Start TTS service

In terminal 2, start the web server to expose the endpoints and UI:

$ uvicorn main:app --host 0.0.0.0 --port 8000

7.Users can now interact with the TTS service

Programmatically using curl requests:

# trigger TTS generation for single article
$ curl -X POST "http://localhost:8000/generate?articles=Earth"

# trigger TTS generation for multiple articles using pipe-separated list just like the MediaWiki action API: https://en.wikipedia.org/w/api.php?action=query&prop=info&titles=Earth|Mars
$ curl -X POST "http://localhost:8000/generate?articles=Earth|Mars"

# get the generated audio for the "Lead" section in the "Earth" article
curl -OJ "http://localhost:8000/audio?article=Earth&section=Lead"

Visually via the browser:

Since Apps team SWEs currently cannot access the prototype hosted on ml-lab, the next step is to deploy this service on toolforge for broader accessibility and testing.

Following T424378#11888422, we started preparing to host the TTS prototype on toolforge. We found that the celery-worker job can currently only run with --mem 4Gi --cpu 2 at most on toolforge because of the resource quotas shown below:

tools.wiki-tts@tools-bastion-15:~$ kubectl describe resourcequotas
Name:                   tool-wiki-tts
Namespace:              tool-wiki-tts
Resource                Used   Hard
--------                ----   ----
configmaps              4      10
count/cronjobs.batch    0      50
count/deployments.apps  1      16
count/jobs.batch        0      15
limits.cpu              500m   16
limits.memory           512Mi  8Gi
persistentvolumeclaims  0      0
pods                    1      16
requests.cpu            125m   16
requests.memory         256Mi  8Gi
secrets                 4      64
services                1      16
services.nodeports      0      0

tools.wiki-tts@tools-bastion-15:~$ toolforge jobs quota
Running jobs                                  Used    Limit
--------------------------------------------  ------  -------
Total running jobs at once (Kubernetes pods)  1       16
Running one-off and cron jobs                 0       15
CPU                                           0.5     16.0
Memory                                        0.5Gi   8.0Gi

Per-job limits    Used    Limit
----------------  ------  -------
CPU                       3.0
Memory                    6.0Gi

Job definitions                             Used    Limit
----------------------------------------  ------  -------
Cron jobs                                      0       50
Continuous jobs (including web services)       1       16

Since we plan to support about 20 concurrent users for this prototype, I ran concurrent benchmarks using benchmark_tts.py. Below are the results from testing 3 resource configurations on ml-lab.

1. 2CPUs and 4GiB with 2 concurrent workers

kevinbazira@ml-lab1002:~/tts$ docker update --cpus="2.0" --memory="4g" bold_montalcini
root@ml-lab1002:/srv/tts_prototype# OMP_NUM_THREADS=1 ORT_NUM_THREADS=1 python3 benchmark_tts.py --concurrent 2

============================================================
  System info
============================================================
  CPU: AMD EPYC 7643P 48-Core Processor
  Arch: X86_64
  INT8 acceleration: AVX2 (no VNNI) — INT8 model may be slower
  Cores: 96
  ONNX Runtime version: 1.25.1
  Available providers: ['AzureExecutionProvider', 'CPUExecutionProvider']

  Benchmark mode: 2-USER CONCURRENT

============================================================
  CONCURRENT BENCHMARK: 2x kokoro-onnx (FP32)
============================================================
  Model: kokoro-v1.0.onnx
  Spawning 2 subprocesses (each loads model independently)

  Per-worker breakdown:
    Worker 0:  400 chars → 18.4s (0.0459 s/char, RTF=0.71x)
    Worker 1:  400 chars → 18.1s (0.0453 s/char, RTF=0.70x)

  Totals:
    Concurrency:      2x
    Wall clock:       20.0s
    Total chars:      800
    Total audio:      51.7s
    Average s/char:   0.0250
    Aggregate RTF:    0.39x
    Throughput:       40 chars/s
    Scaling efficiency: 90% (ideal=concurrency×)

2. 4CPUs and 8GiB with 4 concurrent workers

kevinbazira@ml-lab1002:~/tts$ docker update --cpus="4.0" --memory="8g" bold_montalcini
root@ml-lab1002:/srv/tts_prototype# OMP_NUM_THREADS=1 ORT_NUM_THREADS=1 python3 benchmark_tts.py --concurrent 4

============================================================
  System info
============================================================
  CPU: AMD EPYC 7643P 48-Core Processor
  Arch: X86_64
  INT8 acceleration: AVX2 (no VNNI) — INT8 model may be slower
  Cores: 96
  ONNX Runtime version: 1.25.1
  Available providers: ['AzureExecutionProvider', 'CPUExecutionProvider']

  Benchmark mode: 4-USER CONCURRENT

============================================================
  CONCURRENT BENCHMARK: 4x kokoro-onnx (FP32)
============================================================
  Model: kokoro-v1.0.onnx
  Spawning 4 subprocesses (each loads model independently)

  Per-worker breakdown:
    Worker 0:  400 chars → 18.4s (0.0461 s/char, RTF=0.71x)
    Worker 1:  400 chars → 18.2s (0.0454 s/char, RTF=0.71x)
    Worker 2:  400 chars → 18.3s (0.0457 s/char, RTF=0.71x)
    Worker 3:  400 chars → 18.6s (0.0466 s/char, RTF=0.71x)

  Totals:
    Concurrency:      4x
    Wall clock:       20.3s
    Total chars:      1600
    Total audio:      104.0s
    Average s/char:   0.0127
    Aggregate RTF:    0.20x
    Throughput:       79 chars/s
    Scaling efficiency: 89% (ideal=concurrency×)

3. 8CPUs and 16GiB with 8 concurrent workers

kevinbazira@ml-lab1002:~/tts$ docker update --cpus="8.0" --memory="16g" bold_montalcini
root@ml-lab1002:/srv/tts_prototype# OMP_NUM_THREADS=1 ORT_NUM_THREADS=1 python3 benchmark_tts.py --concurrent 8

============================================================
  System info
============================================================
  CPU: AMD EPYC 7643P 48-Core Processor
  Arch: X86_64
  INT8 acceleration: AVX2 (no VNNI) — INT8 model may be slower
  Cores: 96
  ONNX Runtime version: 1.25.1
  Available providers: ['AzureExecutionProvider', 'CPUExecutionProvider']

  Benchmark mode: 8-USER CONCURRENT

============================================================
  CONCURRENT BENCHMARK: 8x kokoro-onnx (FP32)
============================================================
  Model: kokoro-v1.0.onnx
  Spawning 8 subprocesses (each loads model independently)

  Per-worker breakdown:
    Worker 0:  400 chars → 18.9s (0.0472 s/char, RTF=0.73x)
    Worker 1:  400 chars → 18.6s (0.0466 s/char, RTF=0.73x)
    Worker 2:  400 chars → 18.8s (0.0470 s/char, RTF=0.73x)
    Worker 3:  400 chars → 19.2s (0.0480 s/char, RTF=0.73x)
    Worker 4:  400 chars → 19.1s (0.0478 s/char, RTF=0.73x)
    Worker 5:  400 chars → 18.7s (0.0468 s/char, RTF=0.73x)
    Worker 6:  400 chars → 19.0s (0.0474 s/char, RTF=0.74x)
    Worker 7:  400 chars → 19.0s (0.0476 s/char, RTF=0.73x)

  Totals:
    Concurrency:      8x
    Wall clock:       20.9s
    Total chars:      3200
    Total audio:      207.7s
    Average s/char:   0.0065
    Aggregate RTF:    0.10x
    Throughput:       153 chars/s
    Scaling efficiency: 89% (ideal=concurrency×)

As shown in these benchmark results, to eliminate audio generation queue bottlenecks and provide a smooth, responsive prototype experience, we need to scale the worker to 8 concurrent processes. To achieve this, we'll need 8CPU and 16Gi RAM per-job limit, as well as a corresponding bump to the overall namespace memory limit to support this job running alongside our active FastAPI web service on toolforge.

Workers	CPUs needed	Throughput	Time for 20 × 400-char sections
2	2	40 chars/s	8000 / 40 = 200 seconds
4	4	79 chars/s	8000 / 79 ≈ 101 seconds
8	8	153 chars/s	8000 / 153 ≈ 52 seconds

kevinbazira mentioned this in T425804: Request increased quota for wiki-tts Toolforge tool.May 8 2026, 5:53 PM

kevinbazira added a subtask: T425804: Request increased quota for wiki-tts Toolforge tool.

kevinbazira mentioned this in T425909: Request creation of wikitts VPS project.May 11 2026, 8:00 AM

kevinbazira added a subtask: T425909: Request creation of wikitts VPS project.May 11 2026, 8:02 AM

taavi closed subtask T425804: Request increased quota for wiki-tts Toolforge tool as Resolved.May 14 2026, 4:20 PM

taavi closed subtask T425909: Request creation of wikitts VPS project as Declined.

achou changed the status of subtask T425909: Request creation of wikitts VPS project from Declined to Resolved.May 15 2026, 7:28 AM

achou changed the status of subtask T425909: Request creation of wikitts VPS project from Resolved to Declined.May 15 2026, 7:30 AM

In T424378#11903226, the focus was on vertical scaling. However, after discussions in T425804#11913283 and T425804#11914308, this project was allocated 24Gi RAM to enable horizontal scaling. This approach involves deploying 10 small worker replicas (1CPU and 2Gi RAM each) alongside the web server, as vertical scaling is not currently supported on toolforge. Below are the steps I used to host this TTS prototype on toolforge:

1. SSH into Toolforge and clone the repo

Log into the Toolforge, assume the tool account, and pull the source code.

$ ssh YOUR_USERNAME@login.toolforge.org
$ become wiki-tts

# Create the web directory and clone the repo
$ mkdir -p ~/www/python/src
$ git clone https://gitlab.wikimedia.org/toolforge-repos/wiki-tts.git ~/www/python/src
$ cd ~/www/python/src

2. Install dependencies

2.1. FFmpeg (system dependency)

To compress audio to .mp3 in a restricted Toolforge Linux environment without sudo, we use a static binary for FFmpeg.

# Navigate to local bin (create it if missing)
$ mkdir -p ~/bin && cd ~/bin

# Download the latest FFmpeg amd64 static build
$ wget https://johnvansickle.com/ffmpeg/releases/ffmpeg-release-amd64-static.tar.xz

# Unpack the archive and move binaries to ~/bin
$ tar xvf ffmpeg-release-amd64-static.tar.xz
$ cp ffmpeg-*-amd64-static/ffmpeg ~/bin/
$ cp ffmpeg-*-amd64-static/ffprobe ~/bin/

# Make them executable and clean up
$ chmod +x ~/bin/ffmpeg ~/bin/ffprobe
$ rm -rf ffmpeg-*-amd64-static*

# Confirm successful installation
$ ~/bin/ffmpeg -version

2.2. Python environment

Build the virtual environment inside the Toolforge web shell to ensure C-extensions (like ONNX and hiredis) compile against the correct target OS.

$ toolforge webservice python3.11 shell
$ cd ~/www/python/src
$ python3 -m venv ~/www/python/venv
$ source ~/www/python/venv/bin/activate

# Install dependencies from requirements
$ pip install --upgrade pip wheel
$ pip install -r requirements.txt
$ exit

2.3. Download Kokoro TTS model

Download the optimized Kokoro ONNX model and voice profiles.

$ cd ~/www/python/src
$ wget https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/kokoro-v1.0.onnx
$ wget https://github.com/thewh1teagle/kokoro-onnx/releases/download/model-files-v1.0/voices-v1.0.bin

3. Redis Message Broker

The Celery queue relies on Toolforge's shared Redis. No setup is required. Connection is handled automatically via redis.svc.tools.eqiad1.wikimedia.cloud:6379.

4. Start Celery workers

Start the background inference workers as a continuous Toolforge job. This spans 10 replicas, pinned to 1 CPU thread each, to prevent CPU thrashing and ensure fast parallel generation. (see T425804#11914308)

$ cd ~/www/python/src
$ toolforge jobs run celery-worker \
--command "export ORT_NUM_THREADS=1 OMP_NUM_THREADS=1 MKL_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMEXPR_NUM_THREADS=1 && cd ~/www/python/src && ~/www/python/venv/bin/celery -A wiki_tts.worker worker --pool solo --loglevel=info" \
--image python3.11 \
--continuous \
--replicas 10 \
--mem 2Gi \
--cpu 1

# Confirm the workers are running
$ toolforge jobs list
# Verify all replicas are running. This gives richer status than the jobs list (e.g OOM kills, CrashLoopBackOff, etc)
$ kubectl get pods

5. Start TTS service

Launch the FastAPI application using Toolforge's webservice router.

$ cd ~/www/python/src
$ toolforge webservice python3.11 start

# Confirm the web service is running
$ toolforge webservice status

6. Interacting with the service

6.1. Programmatically via cURL

The primary endpoint that the Apps team will be hitting serves the .mp3 if it exists, or queues it for generation then returns a 202 Accepted status if missing.

# File exists returns HTTP 200 and downloads the .mp3 with the original filename.
$ curl -OJ "https://wiki-tts.toolforge.org/audio?article=Earth&section=Lead"

# File missing returns HTTP 202 instead of 404, queues section generation, and returns JSON status.
$ curl -w "\nHTTP Status: %{http_code}\n" "https://wiki-tts.toolforge.org/audio?article=Earth&section=Atmosphere"

# Section not found returns HTTP 404 with "Section 'FakeSection' not found".
$ curl -w "\nHTTP Status: %{http_code}\n" "https://wiki-tts.toolforge.org/audio?article=Earth&section=FakeSection"

# Article not found returns HTTP 404 with "Article 'NonExistent' not found".
$ curl -w "\nHTTP Status: %{http_code}\n" "https://wiki-tts.toolforge.org/audio?article=NonExistent&section=Lead"

# Generate missing sections of an article.
$ curl -X POST "https://wiki-tts.toolforge.org/generate?articles=Earth"

# Generate missing sections of multiple articles using a pipe-separated list just like the MediaWiki action API: https://en.wikipedia.org/w/api.php?action=query&prop=info&titles=Earth|Mars
$ curl -X POST "https://wiki-tts.toolforge.org/generate?articles=Earth|Mars"

6.2. Visually via the browser

Prototype UI: https://wiki-tts.toolforge.org/
API Documentation: https://wiki-tts.toolforge.org/docs
Prototype Demo: ▶️ F79759624

TheresNoTime subscribed.May 15 2026, 10:57 AM

kevinbazira created subtask T426756: Fix text normalization edge cases in TTS prototype.May 19 2026, 2:25 PM

kevinbazira mentioned this in T426756: Fix text normalization edge cases in TTS prototype.

kevinbazira created subtask T427173: Add section heading announcements in TTS prototype.May 25 2026, 7:10 AM

kevinbazira closed subtask T426756: Fix text normalization edge cases in TTS prototype as Resolved.May 26 2026, 7:01 AM

kevinbazira closed subtask T427173: Add section heading announcements in TTS prototype as Resolved.May 26 2026, 7:31 AM

kevinbazira created subtask T427262: Add static audio URL for native OS audio players in TTS prototype.May 26 2026, 10:51 AM

kevinbazira created subtask T427488: Add word-level timestamps in TTS prototype.May 28 2026, 9:22 AM

kevinbazira created subtask T428435: TTS batch generation pipeline for Featured Articles.Jun 8 2026, 11:18 AM

kevinbazira closed subtask T427488: Add word-level timestamps in TTS prototype as Resolved.

kevinbazira closed subtask T427262: Add static audio URL for native OS audio players in TTS prototype as Resolved.Jun 8 2026, 11:21 AM

kevinbazira updated the task description. (Show Details)Jun 30 2026, 5:12 AM

We built a TTS prototype that provides the following API endpoints:

Endpoint	Method	What it does	Implementation details
`/articles`	GET	List every pre-generated article with section counts.	T428435
`/audio/Earth/Lead.mp3`	GET	Static .mp3 byte-stream for native audio players.	T427262
`/audio/Earth/Lead.vtt`	GET	Word-level WebVTT captions for follow-along highlighting.	T427488
`/generate?articles=Earth│Mars`	POST	Queue article(s) for audio generation. Skips what's already on disk.	T424378#11888422, T424378#11924743, T426756, T427173
`/audio?article=Earth&section=Lead`	GET	Serve the .mp3 if ready (200), queue generation if missing (202), or 404 if invalid.	T424378#11924743, T426756, T427173

Interactive API documentation is available here: https://wiki-tts.toolforge.org/docs

We also added a user interface to this prototype that shows these API endpoints in action:

1.On-demand generation, listening, and follow-along highlighting: https://wiki-tts.toolforge.org

TTS prototype UI: audio generation, listening, and follow-along highlighting.png (1,214×1,569 px, 230 KB)

2.Searchable library of pre-generated articles: https://wiki-tts.toolforge.org/library

TTS prototype UI: search and filter pre-generated audio in library.png (1,254×1,934 px, 210 KB)

The limitations we observed in the v0 prototype that we plan to address in v1 include:

Toolforge hosting provided only 24 GB RAM and CPU, which affects both generation speed and serving throughput for .mp3 files. v1 will use LiftWing GPUs and larger instances.
Audio stored locally by article title without revision tracking. v1 will store by revision ID so audio stays up-to-date with article edits.
Wikipedia plain-text API strips structured content formatting (subscript/superscript like m-squared becomes "m2"). v1 will use HTML-based extraction to preserve structured content (tables, math formulas, etc)
Provides only English audios. Kokoro supports 9 languages. v1 will expand to additional languages iteratively.

Progress on v1 will be tracked in the main task: T419288

kevinbazira closed this task as Resolved.Jun 30 2026, 7:32 AM

kevinbazira mentioned this in T430536: Migrate TTS inference logic from v0 to v1 (without orchestration, storage, and serving layers).

kevinbazira moved this task from In Progress to Done on the Machine-Learning-Team (Q1 FY2026-27) board.

isarantopoulos moved this task from Q1 FY2026-27 to Task Archive on the Machine-Learning-Team board.Jul 6 2026, 7:49 AM

isarantopoulos edited projects, added: Machine-Learning-Team; removed: Machine-Learning-Team (Q1 FY2026-27).

isarantopoulos closed subtask T428435: TTS batch generation pipeline for Featured Articles as Resolved.Aug 11 2026, 2:19 PM

Explore options to run TTS models for evaluationClosed, ResolvedPublicActions