Niiiiiiice! kawiki data is still a little sparse so the error bars on that 4% are big, but this looks really good! Thanks for the update!

@EBernhardson's analysis looks good and explains the discrepency between the original queries (modulo the original incomplete result counts).

An approach using wikidata like T123021 or another language-specific search could be done with the current search APIs. The decisions that need to be made are about configuration, preferences, and UX (searching wikidata or a very short list of wiki-specific projects, etc), which are outside the scope of CirrusSearch.

I thought the other ticket had more information on how to potentially solve the problem and had the right projects tagged since @Jdlrobson-WMF knows more about the issue. I'll update the tags here.

In T331250#11611360, @1234qwer1234qwer4 wrote:

Not sure if this is in any way related, but a generic search such as the one in the screenshot below (-deepcat:"Articles using English type template") would not be expected to return 5 potentially NSFW results within the first 21.

For non-Chinese speakers, you can see this on English Wikipedia if you enable an IME as your input method. I'm using Pinyin on a Mac.

Closing this now that we've done T408734 and friends, and we aren't going to implement this particular approach.

Closing this now that we've done T408734 and friends, and we aren't going to implement this particular approach.

We looked into this a bit and doing it properly is a moderate amount of work, so we are putting it in the backlog for the normal prioritization process.

For now the timeouts are set to reasonable limits, in terms of what our infrastructure can support.

We'd like to work on this at some point, but it is a big task and needs to be prioritized as at least a quarterly project, so making this task an Epic and moving back to the non-sprint workboard.

After thinking about this more while writing up the description, and talking with @dcausse today, I think I'm going to close this ticket for now. If this shows up more often and more annoyingly than it seems to me that it will, someone can re-open the ticket.

Looking at the en/fr/he report and the 20-wiki report.

@Bugreporter, the code you linked to is currently only used for as-you-type suggestions, but your mention of page contents sounds like you are looking for matching in fulltext search results, too. Are thinking of one, the other, or both?

@Pppery, thanks for your bolder edits. I'm always a little hesitant to jump in and make sweeping changes from my WMF account. I appreciate you taking the info I was able to provide and editing it into shape! If you feel like things are done, we can move this ticket to "Done" on the Discovery workboard.

I also added a blurb to next week's Tech News.

Okay, I think I'm done with the updates. (Thanks to @Shirayuki for marking them for translation and making a few other updates, too.)

In T375215#11426047, @stjn wrote:

Is there any plan to make it so that search pages like https://ru.wikipedia.org/wiki/Special:Search/,fhfr_j,fvf («Барак Обама», Barack Obama) display relevant results as well?

I've made some edits and added some information. I will try to finish reviewing the rest of the section tomorrow to soften the language throughout from "never do use a bare regex" to "try no to use bare regexes". (Though if someone beats me to it, I won't complain!)

We're off-topic for this ticket, but I'll reply to these here. @dcausse, do you have the link for the on-wiki help with regex searches you mentioned?

Started looking at this, and it's a little harder to match the style and tone than I expected.

In T410758#11399135, @Taylor wrote:

Indeed it is documented. But how to search for

||}}{{de|

on a wiki where "de" is a common preposition or pronoun? The stupid indexed search will ignore everything except the "de", and since "de" is on every page, it will not generate any "domain".

Maybe wikis would need a simple plain-text search, without all the flawed indexed "magic", and without the resource-hoggy regex magic.

The plugins have been deployed to production CirrusSearch.

Added this:

The Wikimedia Search Team has recreated the "DWIM" gadget functionality server-side, for Russian and Hebrew Wikipedias. This feature appends cross-keyboard suggestions to the standard search-box suggestions. For example, searching for cxfcnmt on Russian Wikipedia will now append suggestions for счастье ("happiness"). We plan to enable this feature for other Russian and Hebrew wikis next week. See also Phabricator T408734 and related tickets.

In T408734#11378210, @dcausse wrote:

could you draft a message for the tech news ...

Moving this to triage, since it is the next step in the NORM/SecondTry project.

Mediawiki notes.

Once this is live, we'll need to reindex. (Added to T408431)

I often open the target folder to find the jar so I can deploy it locally. I also use the Finder with Java projects to recursively open folders to get all the way to the bottom quickly. (Anything that relies on people not opening folders is doomed to fail eventually.)

I believe we should find what is causing them to appear in the first place

As part of T407440: Regexes with four 32-bit characters throw errors I've made some changes to the opensearch-extra plugin which also needs to be deployed. Would it make sense to wait and do them both at once?

Thanks for running all the individual language reports—very interesting to look at them all. I skimmed them, comparing the charts for each to the all_wikis charts.