Following Phase 2 (T432198), the generator produces the full delivery artifact family (audio_opus, audio_mp3, captions_vtt, timestamps_json) with deterministic transcoding and a hardened error taxonomy, but until now it has been running locally on ML-Lab. In this task, we shall make the generator deployable on LiftWing.
- Containerization with the NeMo grammar cache baked at image build. v0 managed NeMo's 60+ second grammar compilation operationally (a manual cache-initialization step, without which k8s readiness probes timed out and pods entered CrashLoopBackOff). We shall bake the compiled grammars into the image as a build step, killing that failure mode permanently: startup will load precompiled grammars in seconds, and the build shall assert NeMo initialized successfully, so an image that would silently fall back to regex normalization (a generation_version change in disguise) fails the build rather than shipping. Layers shall be ordered so the expensive bake caches independently of application-code changes (the known build risk is pynini wheel availability for the base image's Python; if the wheel match breaks, the build grows a compiler toolchain and gets slow, not broken). We shall also migrate the service startup hook to the FastAPI lifespan API and add compose configuration for local development. The production image shall be rebuilt on WMF base images per the deploying repo's conventions; the layer sequence is the property to preserve.
- Staging deployment. We shall deploy the containerized generator to staging and settle, with the team, which cluster a non-KServe ML-owned HTTP service belongs on, along with its resource envelope. The service shall run single-worker and scale via replicas, keeping the concurrency story uniform with the isvc.
- Spike 1: the request-timeout ceiling. Section length maps directly onto single-request duration, so the question is decided by the corpus tail. We shall write a corpus scanner (scripts/scan_corpus.py, committed with both a seeded-sample mode and a full-corpus --all mode) that enumerates the Featured Article list, samples it reproducibly, extracts and normalizes sections through the generator's own code path, and reports the section-length distribution converted to estimated isvc wall time against candidate ceilings (60/120/300/600 s), using the T430536-measured calibration (~15 chars per audio-second, RTF 0.27 full alignment). We shall then measure the scan's worst sections through the real staging isvc (scripts/spike_timeout.py, which shall also recalibrate the chars-per-audio-second and effective end-to-end RTF constants with measured values) and look up the actual LiftWing/Knative timeout configuration on the path. The decision this spike shall produce, in writing: provision/raise the timeout, or implement oversized-section splitting across multiple isvc calls (paragraph-boundary joins with a silence gap); splitting shall not be built unless the measured tail forces it. The scan shall double as a content-quality audit of section extraction and normalization against real corpus text; any defects it surfaces (extraction letting non-spoken content through, normalizer failures on real inputs) shall be fixed with regression tests and the sample rescanned with the same seed, since a distribution computed on buggy extraction would answer the timeout question wrongly, and corrected corpus-level figures (sections per article, total generatable sections) shall be folded into the intake document.
- Spike 2: blob-write mode behind one interface. We shall implement artifact sinks selected by configuration at startup: inline (default, artifacts return as bytes_b64; LAC-native and storage-free), file (atomic tmp-plus-rename writes under the canonical revision-scoped key layout {wiki_id}/{page_id}/{rev_id}/{section_id}.{ext}, response carries blob_uri; the Phase 4 pilot's sink and the local stand-in for object storage), and s3 (interface-complete stub that fails loudly at startup until Data Persistence provisions the PoC bucket). A misconfigured sink shall fail the deploy, not the first request; the response schema shall differ by exactly one field (bytes_b64 vs blob_uri) between modes. This converts the blob-write question from a dependency into "both modes supported; Data Persistence's call."
Acceptance criteria: the image builds with the NeMo bake assertion passing; the service runs in staging; the unit suite is green in CI including sink tests and any scan-surfaced regression tests; SPIKE_ANSWERS.md exists in the repo containing the measured section-length distribution with its sampling caveat, the in-pod worst-case synthesis measurements, the actual infra timeout values on the path, and the written recommendation for each spike; and the Prep Pantry intake document is updated with both answers, with any corrected corpus figures carried into its Operations rows.
Scope of this task: deployment readiness and the two spikes only. Tracked separately:
- Phase 4, batch pilot: the ~50 Featured Article pilot producing the measured numbers pack (real per-file artifact sizes, full-path latency distribution, skip/failure rates by taxonomy code) for the DE intake meeting and the maxReplicas capacity decision; it shall consume this task's file sink and scan tooling and replace the calibration-derived estimates with fully measured values.
- S3 sink wiring: lands together with the Data Persistence PoC bucket and credentials, not before.
- Full-corpus scan: the sampled distribution shall carry a stated extreme-tail caveat; the --all run that replaces the extrapolated corpus maximum with the true one may land here or in Phase 4, whichever executes first.
- DE / Data Persistence / Traffic layers: batch pipeline, edit-stream trigger, generation-events stream, blob store plus index, and serving/CDN remain owned by their respective teams per the intake document.