Change #1298896 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseCirrusSearch@master] Re-enable EntitySearchElasticTest

https://gerrit.wikimedia.org/r/1298896

Change #1298897 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseCirrusSearch@master] Update to align with ResultsType inheritance

https://gerrit.wikimedia.org/r/1298897

In T204089#11990504, @Speravir wrote:

I was going to say that I miss the aspect which was the reason I opened this ticket: finding results without redirects. But from https://gerrit.wikimedia.org/r/1297158 it seems you are aware of this, and I just did not understand the intended scope thing.

It does and it doesn't, i suspect solving that portion will have to build upon what is already planned, but that should be easy once this first, and more difficult, step is done. The challenge with solving this class of problem has been that the search engine did not index and thus could not return redirects as their own search results. The insource queries can't work because the wikitext of redirect pages is not in the index anywhere, the only thing the search engine knew was that page A has redirects [B, C, ...] pointing at it. This first step is introducing the redirect documents to the search index but only exposing them under the new 'redirect scope' where redirects and articles are all equal. This was decided on as the most flexible starting point for building the rest from.

With a query mode that includes redirects as first-class search results it will be much easier to build filters such as returning only redirects, or only redirects that point at a specific page, or a specific namespace. Often times the hardest part of building those filters, when the search index already supports them, is defining the name and semantics of the keywords. It's often unclear how much we should bake in though, as insource:/.../ should be able to answer many more questions than we can imagine. Are there any new semantics that would serve your use cases better than insource?

This said, since I opened this ticket almost 8 years are gone, and something’s seems to have changed: The last examples with a search for "Scil" seem to work, now. But there is another inconsistency I do not get at all: The first two examples still work like 8 years ago, but the third with search for "postdam" does not! (Edit: See update.) And that’s most annoying. The search result still displays the true issues, and still also displays the pages where the issues has actually been fixed, but I do not get part with the hint “Redirected from“ (I get the German translation, though, on search queries where it works), the one embedded in a <span class="searchalttitle">…</span>. I remember having this non-display issue for other search queries, too. In fact, I was surprised that the first two search examples still display this part.

Update: Oh, if I change the sort criterion for the "postdam" search query (in fact to alphabetic), the redirect string is suddenly displayed, again.

It looks like the highlighting variance is perhaps an issue with MediaSearch we need to look into. The query working when set to alphabetic, but not showing the highlights when set to relevance sort suggests that the specialized media searching query is not exposing the search terms to the highlighting layer in the same way as the default cirrus query. I see the same general behavior with your other avaition example. This will always be awkward in the default mode though, two words can match on two different redirects and the search index doesn't really know, I'm hoping though that the redirect scope will better serve this use case by allowing to search the redirect documents directly and avoiding this semantic overlap.

Change #1299603 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseMediaInfo@master] Skip MediaSearchQueryBuilderTest

https://gerrit.wikimedia.org/r/1299603

Change #1299604 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/WikibaseMediaInfo@master] Re-enable MediaSearchQueryBuilderTest

https://gerrit.wikimedia.org/r/1299604

Change #1297157 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Add page_type and redirect_target fields

https://gerrit.wikimedia.org/r/1297157

Change #1297157 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Add page_type and redirect_target fields

https://gerrit.wikimedia.org/r/1297157

Change #1299603 merged by jenkins-bot:

[mediawiki/extensions/WikibaseMediaInfo@master] Skip MediaSearchQueryBuilderTest

https://gerrit.wikimedia.org/r/1299603

Change #1298877 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Skip MoreLikeWikibaseTest

https://gerrit.wikimedia.org/r/1298877

Change #1298895 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@master] Skip EntitySearchElasticTest

https://gerrit.wikimedia.org/r/1298895

Change #1297158 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Add redirectScope concept to SearchContext

https://gerrit.wikimedia.org/r/1297158

Change #1300939 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] LexemeFullTextQueryBuilderTest: Mark test as skipped

https://gerrit.wikimedia.org/r/1300939

Change #1300940 had a related patch set uploaded (by Reedy; author: Reedy):

[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] Re-enable LexemeFullTextQueryBuilderTest

https://gerrit.wikimedia.org/r/1300940

Change #1300939 merged by jenkins-bot:

[mediawiki/extensions/WikibaseLexemeCirrusSearch@master] LexemeFullTextQueryBuilderTest: Mark test as skipped

https://gerrit.wikimedia.org/r/1300939

Change #1298896 merged by jenkins-bot:

[mediawiki/extensions/WikibaseCirrusSearch@master] Re-enable EntitySearchElasticTest

https://gerrit.wikimedia.org/r/1298896

Change #1301427 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] ForceSearchIndex: build redirect documents during reindex

https://gerrit.wikimedia.org/r/1301427

Change #1301428 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] saneitizer: Handle first-class redirect documents

https://gerrit.wikimedia.org/r/1301428

Change #1299604 merged by jenkins-bot:

[mediawiki/extensions/WikibaseMediaInfo@master] Re-enable MediaSearchQueryBuilderTest

https://gerrit.wikimedia.org/r/1299604

Change #1298897 abandoned by Ebernhardson:

[mediawiki/extensions/WikibaseCirrusSearch@master] Update to align with ResultsType inheritance

Reason:

Went with a different solution which kept the interface unchanged

https://gerrit.wikimedia.org/r/1298897

Change #1297160 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Wire redirect documents into the update process

https://gerrit.wikimedia.org/r/1297160

Change #1301427 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] ForceSearchIndex: build redirect documents during reindex

https://gerrit.wikimedia.org/r/1301427

ebernhardson opened https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-primary/-/merge_requests/62

cirrus: Add fields for first class redirects

ebernhardson opened https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/206

first-class redirects: update schema

ebernhardson merged https://gitlab.wikimedia.org/repos/search-platform/cirrus-streaming-updater/-/merge_requests/206

first-class redirects: update schema

ebernhardson merged https://gitlab.wikimedia.org/repos/data-engineering/schemas-event-primary/-/merge_requests/62

cirrus: Add fields for first class redirects

Change #1301428 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] saneitizer: Handle first-class redirect documents

https://gerrit.wikimedia.org/r/1301428

https://meta.wikimedia.org/wiki/Community_Wishlist/W356

Change #1298808 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] first-class-redirects: Wire up query side

https://gerrit.wikimedia.org/r/1298808

In-place reindexes to add the new fields are in progress. Likely next week we can flip the saneitizer feature flag which will make the redirects be seen as missing documents and repaired. That will take a few more weeks once enabled to load the redirects across all projects.

reindexes are completed in eqiad and codfw, but cloudelastic is struggling to complete the tasks. This most likely has more to do with the recent upgrade from opensearch 1.x to 2.x, than anything specific to this change, but will delay things by a little bit while we figure out why it's not happy.

Change #1320212 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Set reindex slices to shard count

https://gerrit.wikimedia.org/r/1320212

ebernhardson opened https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2462

search: Add new cirrus fields to table definition

Change #1320230 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/mediawiki-config@master] cirrus: Enable building redirect documents

https://gerrit.wikimedia.org/r/1320230

Change #1320212 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Set reindex slices to shard count

https://gerrit.wikimedia.org/r/1320212

dcausse merged https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/2462

search: Add new cirrus fields to table definition

Change #1321612 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Gate QueryCirrusDoc redirectscope on build flag

https://gerrit.wikimedia.org/r/1321612

Change #1321612 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Gate QueryCirrusDoc redirectscope on build flag

https://gerrit.wikimedia.org/r/1321612

Change #1322039 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[operations/deployment-charts@master] cirrus sup: Enable first class redirect handling

https://gerrit.wikimedia.org/r/1322039

Change #1320230 merged by jenkins-bot:

[operations/mediawiki-config@master] cirrus: Enable building redirect documents

https://gerrit.wikimedia.org/r/1320230

Mentioned in SAL (#wikimedia-operations) [2026-08-06T20:24:25Z] <cjming@deploy1003> Started scap sync-world: Backport for [[gerrit:1319939|Revert "frwiki: change to Wikipedia 25 logo" (T430409)]], [[gerrit:1320230|cirrus: Enable building redirect documents (T204089)]]

Mentioned in SAL (#wikimedia-operations) [2026-08-06T20:26:15Z] <cjming@deploy1003> cjming, ebernhardson, chlod: Backport for [[gerrit:1319939|Revert "frwiki: change to Wikipedia 25 logo" (T430409)]], [[gerrit:1320230|cirrus: Enable building redirect documents (T204089)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.

Change #1322039 merged by jenkins-bot:

[operations/deployment-charts@master] cirrus sup: Enable first class redirect handling

https://gerrit.wikimedia.org/r/1322039

Mentioned in SAL (#wikimedia-operations) [2026-08-06T20:31:07Z] <cjming@deploy1003> Finished scap sync-world: Backport for [[gerrit:1319939|Revert "frwiki: change to Wikipedia 25 logo" (T430409)]], [[gerrit:1320230|cirrus: Enable building redirect documents (T204089)]] (duration: 06m 41s)

Change #1327189 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Implement onlyredirects: keyword

https://gerrit.wikimedia.org/r/1327189

Change #1327189 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Implement onlyredirects: keyword

https://gerrit.wikimedia.org/r/1327189

Change #1328701 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/core@master] search: Use redirect=no when linking a redirect

https://gerrit.wikimedia.org/r/1328701

Change #1328701 merged by jenkins-bot:

[mediawiki/core@master] search: Use redirect=no when linking a redirect

https://gerrit.wikimedia.org/r/1328701

This looks to be working as expected. Using english wikipedia as an example, 12M redirects in content namespaces and 15M redirects across all namespaces are now searchable using typical keywords like insource://, intitle://, etc.

Next steps require documenting these new keywords on mw.org, linking that from a tech news post, and potentially followups collecting use cases for other keywords that might be useful for querying redirects specifically. The search index breaks out the redirect target into interwiki/namespace/title/fragment, but those specific fields are not queried by anything today.

For anyone potentially as confused as I was: to make it work, onlyredirects: has to be the first thing in the search request, and only then should any keywords go. For some reason, intitle:"а" onlyredirects: returns nothing while onlyredirects: intitle:"а" correctly returns 73 results. Thank you to everyone who made this possible :-)

Indeed onlyredirects: (search only against redirects) and withredirects: (combined search against all known pages) have to be a query prefix, this was done for a few reasons that i will try to properly document.

• This stays consistent with the existing pattern on keywords like local: where a valueless keyword that changes the state of the whole query has to be a prefix. The initial intent here was for it to be clear that local: applies to the whole search, otherwise someone could intuit that a local: b will search a everywhere, and only b would be constrained to local.
• An internal implementation detail means we need to parse (only|with)redirects: prior to intitle://, and parsing happens left to right.

Looking at the behaviour when searching a onlyredirects: it seems we need a warning to tell the user what is going on. I don't remember why local: doesn't issue a warning, I'll look into adding one so it's less surprising what happened when the keyword isn't a prefix.

Change #1330643 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] warn when ignoring a keyword that requires being a queryHeader

https://gerrit.wikimedia.org/r/1330643

@EBernhardson, I am responding to the usernotice tag. Please provide an entry to be added to the tech news draft.

I added some docs, at least as a starting point, to Help:CirrusSearch. I suspect this will need to be further refined a bit, it feels unclear but I'm a bit too close to the project to pin down what exactly. Will seek some review.

@STei-WMF

I'm thinking something like the following could work

• Two new search keywords, onlyredirects: and withredirects:, expose redirect's as their own distinct searchable pages. These keywords can be combined with most existing keywords and sort orders such as incategory:, intitle:, and insource:.

So is the exclusion still not implemented? Or has the default behavior changed so that a search doesn't cover redirects unless prefixed with withredirects:?

@EBernhardson can we add more details?:

Who is the target audience for this entry? – I believe this would be readers and editors searching for pages

What exactly is happening? – Two new search keywords, onlyredirects: and withredirects: are now available, they expose redirects as their own distinct searchable pages. These keywords can be combined with most existing keywords and sort orders such as incategory:, intitle:, and insource:.

Why is it changing, or how is it an improvement? What problem does this solve for users? – This isn't clear in the entry.

How should users react? What action they should take? – Correct me if I am wrong but I don't see any action needed other than being aware of these new keywords.

Where can they learn more? – You already provided the links

When is the change happening? It seems this has happened already, we need to tweak the tense of the sentence.

Okay, so now half of this task has been done, to be able to parse the redirect pages themselves. But the other half, to be able to search while excluding redirects hasn't, right?

In T204089#12264500, @STei-WMF wrote:

@EBernhardson can we add more details?:

Who is the target audience for this entry? – I believe this would be readers and editors searching for pages

Target is predominantly editors, particularly editors that edit/curate redirect pages. We generally believe that the defaults were appropriate for readers to whom redirects are a small detail, a piece of the relevance matching. Exposing the redirects as their own pages allows editors to target them with the existing keywords like insource, incategory, hastemplate, etc. Previously the templates, categories, source text, and other metadata for redirect pages simply didn't exist inside the search engine.

What exactly is happening? – Two new search keywords, onlyredirects: and withredirects: are now available, they expose redirects as their own distinct searchable pages. These keywords can be combined with most existing keywords and sort orders such as incategory:, intitle:, and insource:.

For what exactly is happening it really depends on the perspective, from the dev side the main thing is we added millions of previously unsearchable pages to the search engine. But from the editors side it seems to me it is more relevant to call our the new keywords that allow them to access those pages via the search engine, rather than talking much about the pages themselves.

Why is it changing, or how is it an improvement? What problem does this solve for users? – This isn't clear in the entry.

I'm not sure how to make it clear, whats happening is before users could not search the redirects pages. The only thing that was searchable about redirects was the title string, and that title string was attached to the redirected-to page, not the redirect itself. For example:

• this change added 15M new previously unsearchable redirect pages to the enwiki search indexes. Same effect on all other wikis, but different numbers.
• it was previously not possible to find redirects in a particular category or with a particular template and combine that with other arbitrary search keywords and sort orders. The search engine did not have a dedicated page in the index for the redirect, there was nowhere to query those from.
• template and category are just examples, we should now have all the metadata for redirects in the search engine. If we missed some metadata there is now a direct place to add that.

How should users react? What action they should take? – Correct me if I am wrong but I don't see any action needed other than being aware of these new keywords.

It really depends on the editors use cases. I suspect, without being deeply aware, that these search keywords are relevant to a subset of editors who edit redirect pages. Similarly for editors that maintain templates and categories that are used on redirect pages. I suppose the high level view might be that this should improve the ability to curate redirects.

Where can they learn more? – You already provided the links

When is the change happening? It seems this has happened already, we need to tweak the tense of the sentence.

Yes this is now live on all our wikis as of this weeks deployment.

In T204089#12264167, @Nardog wrote:

So is the exclusion still not implemented? Or has the default behavior changed so that a search doesn't cover redirects unless prefixed with withredirects:?

In T204089#12265340, @Jonteemil wrote:

Okay, so now half of this task has been done, to be able to parse the redirect pages themselves. But the other half, to be able to search while excluding redirects hasn't, right?

My appologies, i suppose i've been so focused on the hard part of this task that i didn't get to this side. This is very easy to implement, the main question is how should it be exposed? What keyword or query adjustment makes it happen?

If it’s another start-only keyword like onlyredirects:/withredirects:, I suppose it should be noredirects: for the third one?

@EBernhardson, thank you!

Change #1331719 had a related patch set uploaded (by Ebernhardson; author: Ebernhardson):

[mediawiki/extensions/CirrusSearch@master] Introduce noredirects: keyword

https://gerrit.wikimedia.org/r/1331719

In T204089#12265962, @stjn wrote:

If it’s another start-only keyword like onlyredirects:/withredirects:, I suppose it should be noredirects: for the third one?

Perfect.

Change #1331719 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] Introduce noredirects: keyword

https://gerrit.wikimedia.org/r/1331719

Change #1330643 merged by jenkins-bot:

[mediawiki/extensions/CirrusSearch@master] warn when ignoring a keyword that requires being a queryHeader

https://gerrit.wikimedia.org/r/1330643

I've drafted a thread for WP:VPT to collect use cases for more redirect specific use cases. We should still think about how to collect these usecases outside enwiki though. Intending to post this on monday, I didn't want to put it up on friday before i disappear for the weekend.

Posted: https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Curating_redirects_via_search

Change #1298878 merged by jenkins-bot:

[mediawiki/extensions/Wikibase@master] Restore MoreLikeWikibaseTest

https://gerrit.wikimedia.org/r/1298878