Related MLR spike work: T385971: [SPIKE] Can the results of abandoned query analysis be used to improve MLR performance?

Current write-up—including tags & stats for the English Wikipedia sample, and lots, lots more details—on MediaWiki.

I've finished the English analysis, which included coming up with an annotation scheme for grappling with the queries and the scant information we can glean from them. The full list of goals in this ticket is too ambitious: user intent and search "success" are, in general, notoriously hard to pin down, and in this data isn't always possible to discern the searcher's intent, let alone their reason for abandoning their query.

I struggled at first to even find any patterns in the data that I could identify and label, but retracing the search sessions helped me understand what searchers were probably thinking about and searching for. I settled on labelling fairly objective attributes of searches, suggestions, results, and searcher behavior.

There's no glaring, smack-you-in-the-face solution to query abandonment. Some features of abandoned queries and search sessions:

• People are most often searching "well", that is, looking for something that looks like an article title—though questions, complex queries, etc. do certainly occur.
• Sometimes the info they want just isn't in Wikipedia—there's no article on that thing. Sometimes, the best piece of info about a thing is on the page they are searching from (i.e., they are looking for more info, but there is no more info.)
• Sometimes people ignore good autocomplete suggestions, which is sad when they then get poor fulltext results.
• Unclear snippets can hide good results.
• Google, without notability requirements or limits to encyclopedic content, often has more information.
  • Google's AI Overview does not help nearly as often its fulltext results. Mostly it helps on queries that look more like phrases or sentences than keywords.
  • I've noticed that DuckDuckGo has a smaller index than Google. I have not checked how often it gets results.

I've noted lots of potential improvements to search, autocomplete, snippets, and even query mining. Not all are directly related to abandoned queries. Some highlights:

• Don't show DYM suggestion or results for DYM suggestion when there are no results!
• Use page-specific stats for snippet selection.
• Better formatting for lists in snippets.
• Make section anchors work from autocomplete suggestions.
• Create a "text-only" title index for the Go feature.
• Think again about showing cross-language "round-trip" results.
• Thesaurus!
• Maybe mine abandoned queries—they seem to be a better source of mining for possible new articles or redirects than zero-results queries. (All the usual caveats about query mining still apply.)

I've finished reviewing the Spanish abandoned samples. Very broadly, the tags stats are roughly the same, with a few standouts:

• Spanish searchers were more likely to search from an article page than the main or search page.
• Spanish searchers were more likely to get no suggestions for their full query
• Spanish searchers were more likely to abandon their query even though a page with the info they probably needed did exist
• Google was less likely to have the answers they wanted

Observations and Ideas:

• DYM suggestions in Spanish were more likely to look kinda dumb because they are ungrammatical, or they are essentially the same query because all that got changed was grammatical endings on words.
• We could in theory do a little better surfacing better sister search results and results from the wikidata widget, but the math and/or UI is hard.

More details in the full write up (search for eswiki and Spanish or check the diffs).

Whew, done at last! I've finished the analysis of French, and did a super brief review of just the queries themselves for German, Italian, Portuguese, and Russian. (Our old friends the wrong-keyboard queries showed up in Russian!)

Abbreviated Summary

There's no obvious single smoking gun to explain why searchers abandon their fulltext queries, which is not really a surprise. English, Spanish, and French have very broadly similar patterns of queries and detectable searcher behavior. German, Italian, Portuguese, and Russian do not have obviously wildly different patterns of queries, though Russian does have more queries that are not just straightforward "human queries in the home language of the wiki".

There are a lot of well-formed queries by people who seem to understand how to search Wikipedia. Sometimes they get results, sometimes not. Some people don't know how to search particularly well, but sometimes these sub-optimal queries still get pretty good search results.

Our notion of "abandoning" a fulltext query may need some refinement, since our autocomplete is so good and handles so much of our search traffic. Some users clearly navigate away from "abandoned" fulltext queries by using autocomplete to take them to the next page (or several pages) of interest.

• We might want to reconsider whether we should count a search session as abandoned if the searcher clicks on an autocomplete suggestion after a fulltext search.
• If we look at abandoned queries again, we might want to try to get a more complete version of the searcher/reader's path through our pages.

General improvements to search quality, query handling, second-try searching, and presentation may help people find information more easily, but we are always going to have people just ending their wiki spree on the search page.

A few new thoughts:

• Sometimes snippets can give "hints" to searchers who have typos or other problems in their queries, but it is happenstance and unreliable.
• Wikidata (possibly via the Wikidata widget) is also really helpful when there is a lot of variation in a name. (Example of 19th century nobleman named Karl, Carl, Charles, Carlos, Carlo, Karel, or Karol, depending on the language... and most don't bother to list his name in his native language!)
  • What other cool ideas have been implemented as widgets or gadgets? French provides cross-language and cross-project links if you get zero results, for example!
• Matching category titles (exact or near exact) might help for some category-like searches, e.g., Ukrainian musicians

More details in the full write up (search for frwiki and French, see the de/it/pt/ru section, and the summary.. or check the diffs).