There are different options. Claude proposed a few, see below. We may have access to excess ML compute resources this quarter, see slack thread.

Highlighting for Semantic Search Results — Options & Phased Design

Context

CirrusSearch's semantic search path (neural query against nested passage_chunk_embedding.knn, per-paragraph embeddings) currently shows the entire matched paragraph as the snippet: SemanticSearchResultBuilder::doInnerHits() copies the inner hit's _source[text] verbatim, and SemanticResultsType::getHighlightingConfiguration() returns null. There is no sub-paragraph focusing and no searchmatch emphasis, unlike lexical search (WMF experimental highlighter plugin). OpenSearch's built-in semantic highlighter was evaluated with poor results — explainable: its stock model (opensearch-semantic-highlighter-v1) is a sentence classifier trained on English-only extractive-QA data. SemanticSearchQueryBuilder already carries a TODO: highlighting.

Constraints (confirmed): multilingual from day one; no model inference inside the search cluster (Lift Wing via ML Commons remote connectors is fine); snippets in-band with the search response; fine-tuning a custom model is an option; embeddings come from an Airflow/Spark bulk job (→ Flink later) so index-time data is fully controllable; embedding model is Qwen3 → jina-embeddings-v5-text-small next quarter; semantic search runs on a dedicated OpenSearch 3.5 cluster (the lexical cluster, currently 2.19.5, migrates to 3.7 soon) — so 3.x-only features (semantic-highlighter framework, batch inference ≥3.1) are available for semantic search today, and a custom plugin there is decoupled from the lexical serving path.

Options

Where the implementation lives (direct answer)

• CirrusSearch — the orchestration/presentation seam: highlight config on the semantic path, snippet assembly, profiles + config globals. Thin.
• Embedding pipeline (Spark→Flink) — sentence segmentation, sentence vectors/offsets. The multilingual heavy lifting, at index time.
• Lift Wing + ML Commons connector config — only for Option B (phase 2).
• A vector-highlighter plugin (new 3.x-based module, deployed on the dedicated semantic cluster) is the candidate production form of the query-time step — viable now that semantic search runs on 3.5 (no 2→3 churn; deployment risk contained away from the lexical path). It is *not* an extension of the experimental highlighter (lexical/postings machinery, no shared code). Sequencing: validate snippet quality with the PHP variant first (days, not a plugin release cycle); adopt the plugin if its spike is clean.

Phase 0 — Offline evaluation harness (parallel track; decides how far to take Option B)

Lives in the relevance-eval tooling, not CirrusSearch. Eval set (query, page_id, passage_text, sentence_spans[], gold_sentence_idx[]), ~200–500 items over ≥5 typologically diverse languages (e.g. en/de/ar/ja/hi); sources: semantic-search log queries joined with clicks (weak labels) + TyDi QA / MLQA / XQuAD (Wikipedia passages, answer spans → gold sentences). Candidates: (a) query-vs-sentence cosine with production model (run Qwen3 and jina v5); (b) bge-reranker-v2-m3 cross-encoder; (c) stock opensearch-semantic-highlighter-v1 (quantify the known-poor baseline); (d) lexical-overlap (what Phase 1 ships); (e) optional fine-tuned multilingual sentence classifier. Metrics: sentence P@1, MRR, token-F1, per language. Decision rule: if (a) ≈ (b)/(e), Option A is the end state; if a model clearly wins, Option A ships as interim and B becomes the target. Phase 1 ships regardless. Since the semantic cluster already runs 3.5, a cheap side-spike: re-run the built-in semantic highlighter with a *remote multilingual* model (even an unfine-tuned one) to separate framework problems from model problems in the earlier poor results.

Phase 1 — Baseline: lexical highlighting on the semantic path (CirrusSearch only)

Design decision: highlight inside inner_hits (Elastica InnerHits::setHighlight() exists) rather than the top-level text field, so the snippet is always the semantically matched passage and stays consistent with the section anchor. Requires passage_chunk_embedding.text mapped as analyzed text (coordinate with the pipeline owners; verify current mapping). Implement main_text mode too as a profile-selectable stopgap if the mapping change lags.

Changes:

1. [SemanticSearchQueryBuilder.php](includes/Query/SemanticSearchQueryBuilder.php) — new profile settings highlight.mode (none|inner_hits|main_text, later sentence_vector|semantic), highlight.type, number_of_fragments (default 0 = whole passage with markers). In buildQuery(), for inner_hits mode attach a highlight config to the InnerHits object: pre/post tags Searcher::HIGHLIGHT_PRE_MARKER/POST_MARKER, field {nested}.{snippet_field}, and an explicit per-field highlight_query (match on the term) — mandatory because the neural/knn query yields no extractable terms. Add a snippet_field setting so the builder and SemanticResultsType share profile keys. For main_text mode call $searchContext->setHighlightQuery().
2. [SemanticResultsType.php](includes/Search/SemanticResultsType.php) — for main_text mode accept a FetchPhaseConfigBuilder (available in Searcher::search()) and build a standard config via newHighlightField('text', TARGET_MAIN_SNIPPET); keep returning null for inner_hits mode (config lives inside inner_hits). Pass the highlight settings down to the result builder.
3. [SemanticSearchResultBuilder.php](includes/Search/SemanticSearchResultBuilder.php) — in doInnerHits(): prefer hit['highlight']["{nested}.{snippet}"][0] (when containsMatches()) over raw source, run through escapeHighlightedText() (markers → <span class="searchmatch">), record provenance via textSnippetField(). Also escape the raw-source fallback — today it's emitted unescaped; this is a latent HTML-injection/consistency bug and gets fixed here.
4. [Searcher.php](includes/Searcher.php) (~line 221) — pass the fetch phase builder into SemanticResultsType.

Phase 2 — Option A: sentence-level selection, no query-time inference

Data model (pipeline + mapping; piggyback on the Qwen3→jina reindex)

Extend each nested passage doc — all source-only (no index, no doc values, no second-level nested, no HNSW):

• embedding_version: keyword — model+revision+dims (guards against mixed-version docs during migration);
• sentence_offsets: int[] — flattened [start0,end0,…] byte offsets into text (pin the bytes-vs-codepoints contract with the segmenter; PHP slices with substr());
• sentence_vectors: binary — one base64 blob, n_sentences × dims int8, matryoshka-truncated to ~128 dims (Phase 0 measures the quality delta; sentence argmax is very quantization-tolerant).

Size: ≈ 13 KB/page of _source at P=12 passages × S=6 sentences × 128 dims (~40 GB pre-compression for a 3M-page wiki). No graph/inverted-index cost.

Query-time wiring (variant A1-php — recommended)

1. Query embedding in CirrusSearch: new includes/Search/Embedding/QueryEmbeddingService (interface) + MlCommonsQueryEmbeddingService calling POST /_plugins/_ml/models/{id}/_predict through the existing Elastica client — same cluster endpoint, ML Commons routes to the Lift Wing remote connector (inference stays off-cluster). Hard timeout (~500 ms), WANObjectCache keyed on hash(model_id . instructions . term) (~1 h TTL). Wire via ServiceWiring; reach the builder by extending Searcher::buildFullTextBuilder().
2. Switch neural → raw knn query (new includes/Elastica/KnnQuery.php) with the obtained vector, so the model runs once for retrieval *and* snippets. Net latency ≈ zero (explicit _predict replaces in-query inference; one extra in-DC round trip; cache makes repeats cheaper than today). Keep the NeuralQuery path as degraded mode: embed() → null ⇒ today's query, Phase-1 lexical highlighting still applies.
3. Sentence selection in PHP: add the three new fields to inner_hits _source; new pure class includes/Search/Embedding/SentenceSelector.php — decode blob, dot-product against the (truncated) query vector, argmax, slice by offsets, wrap the winning sentence in the highlight markers, return passage. Thread the query vector via a SearchContext::setSemanticQueryVector()/get…() pair (mirrors setHighlightQuery). Preference cascade in doInnerHits(): sentence-vector (if vectors present + embedding_version matches) → lexical highlight → escaped raw passage. Cost: ~25 KB extra payload per 21-result response; ~16k multiply-adds in PHP (≪1 ms).

(Variant A1-server — second-level nested knn_vector with exact-score script_score inner_hits — documented in the working notes as fallback if payload ever matters; weaker due to score pollution and nested-in-nested reconciliation.)

Variant A2 — vector-highlighter plugin (server-side production form; spike-gated)

Same index-time data model; instead of shipping vectors in _source, a custom highlighter (new Java plugin module built against 3.x, deployed only on the dedicated semantic cluster) extracts the query vector from the rewritten knn query, dot-products the stored sentence vectors server-side, and returns the passage with the winning sentence wrapped in HIGHLIGHT_PRE_MARKER/POST_MARKER — HighlightingTrait and the doInnerHits() cascade consume it unchanged. Retrieval keeps the neural query: no _predict pre-call, no neural→raw-knn switch, no score-invariance validation, and snippets become a property of the index (available to non-MediaWiki consumers). Prerequisite spike (1–2 days on 3.5): (a) custom-highlighter behavior inside nested inner_hits; (b) query-vector extraction from the rewritten query in the block-join context. Decision rule: run A1-php as the quality-experiment vehicle; if Phase 0 + relforge confirm embedding-similarity snippets and the spike is clean, port the selection step into the plugin for production and keep A1-php code as the degraded mode.

Phase 3 — Option B: semantic highlighter framework + custom multilingual model (eval-gated; framework available now on the 3.5 semantic cluster)

Preconditions: Phase 0 shows a model beats embedding-similarity enough to justify per-SERP GPU inference; fine-tuned multilingual sentence/span model (training data: TyDi/MLQA/XQuAD + translated MultiSpanQA + weak labels from retrieval) served on Lift Wing, registered as an ML Commons remote-connector model.

CirrusSearch changes are small by design: new includes/Search/Fetch/SemanticHighlightedField.php emitting ['type' => 'semantic'] with options: {model_id, batch_inference}; mode: semantic in the profile; cascade becomes semantic → sentence-vector → lexical → escaped raw so a degraded model endpoint automatically falls through and search never fails on highlighting. Spike items: nested-field/inner_hits support of the 3.x semantic highlighter; failure semantics on model timeout (must not fail the shard).

Config/profile surface

Behavior in profiles, deployment facts in globals (repo convention):

• Semantic FT_QUERY_BUILDER profile gains highlight: {mode, type, number_of_fragments, sentence_offsets_field, sentence_vectors_field, vector_dims} and embedding: {source: neural|predict, cache_ttl}.
• New globals in extension.json + docs/settings.txt: CirrusSearchSemanticEmbeddingModelId (map cluster → ML Commons model id; [] = keep neural/index-default; Phase 2), CirrusSearchSemanticHighlighterModelId (Phase 3).
• No new hooks; SearchProfileServiceFactory::loadSemanticSearch() unchanged.

Verification

• Phase 1 unit/integration (existing test files): tests/phpunit/unit/Query/SemanticSearchQueryBuilderTest.php — inner_hits highlight array shape, main_text sets highlight query, mode: none byte-identical to today; tests/phpunit/integration/Search/SemanticSearchResultBuilderTest.php — highlight-preferred-over-source, marker conversion, escaping of raw fallback incl. <script> payload, provenance; SemanticResultsTypeTest.php — config per mode.
• Phase 2: pure-PHP tests for SentenceSelector (blob decode, argmax, multibyte/combining-char offset slicing) and MlCommonsQueryEmbeddingService (mocked client: success/timeout/cache); builder emits raw knn with vector vs neural fallback. Cluster-dependent paths: relforge smoke test (knn + ml-commons + stub connector), not CI. Validate neural→knn score-invariance with paired queries on relforge before flipping.
• Dark launch: semantic route is already gated on cirrusSemanticSearch=1; add cirrusSemanticHighlight=<mode> to CirrusDebugOptions for per-request mode override → side-by-side snippet comparison on identical queries. Log textSnippetField provenance into event logging for CTR-per-snippet-source once semantic search itself is A/B-tested. Inspect requests with cirrusDumpQuery/cirrusDumpResult.
• Phase 3: staged rollout relforge → low-traffic wiki, watching model error rate and search P95; mode revertible by config alone.

Risks / open questions

1. Unescaped raw snippet today — treat as bug fix in Phase 1; confirm consumers treat snippets as HTML (they do for full-text).
2. Mapping: is passage_chunk_embedding.text currently an analyzed text field? Inner-hits highlighting needs it; requires a coordinated reindex.
3. Verify plain-highlighter semantics inside inner_hits: number_of_fragments: 0 + per-field highlight_query returns the whole field with marks, and returns *nothing* on no match (the fallback depends on it).
4. Byte-offset contract between segmenter (Spark/Flink) and PHP substr() — a mismatch corrupts snippets precisely on non-Latin wikis.
5. Model/dims skew during Qwen3→jina window — embedding_version guard is mandatory.
6. _predict access from the CirrusSearch client principal + first-hit (cold) latency on Lift Wing vs the tens-of-ms budget.
7. Option B unknowns: nested-field support, failure semantics, 3.1 batch-inference dependency, fine-tuning + hosting cost.