CirrusSearch should be adapted to emit a highlight request of type semantic.
Approach could be to re-use the current semantic debug flag to make highlighting opt-in.
Query shape:
{ "size": 5, "_source": [ "title", "page_id" ], "query": { "nested": { "path": "passage_chunk_embedding", "score_mode": "max", "query": { "neural": { "passage_chunk_embedding.knn": { "query_text": "__QUERY__", "k": 10 } } }, "inner_hits": { "size": 1, "_source": { "excludes": [ "passage_chunk_embedding.knn" ] }, "highlight": { "fields": { "passage_chunk_embedding.text": { "type": "semantic" } } } } } }, "ext": { "semantic_highlighting_batch": true } }
The semantic highlighter model_id will be set on the search pipeline.
AC:
- CirrusSearch is able to emit a semantic highlighter query and feed the snippets in the search response
- highlighting should be opt-in and decided by the client
- https://gitlab.wikimedia.org/repos/search-platform/cirrus-toolbox is adapted to update the search pipeline with a semantic highlighter model id using the semantic_highlighter_query_enricher processor.