As a product manager, I want to understand the breakdown of user search traffic on emerging language wikipedias, so I can understand the estimated scale of impact of planned features as part of special:search experimentations.

Background and context

Search and Structured Data teams plan to work on improving special:search experience on emerging language wikis that generally have less content/articles than bigger wikipedias
Use case: If there is no exact article match in the go bar, the reader is redirected to the special search page. Improve the user experience on the special search page by showing more/relevant content.
Users: Casual readers on emerging language wikipedias.

Previous analysis: There was an analysis in 2017 that produced the following diagram of results:

In order to have an understanding of the volume of potential impact, we would like to answer the following high level questions: How many users end up on the special search page because there is no exact article match for what they are looking for? Does that happen often?

In order to further establish baselines of search performance, we would like to understand search engagement when getting to the special search page (clicks-thru-rate, etc..)

Requirements

Examples of questions we would like to answer:

• Total search volume per wiki: What is the total number of searches in the go bar?
• Autocomplete only
• Go bar-to-special:search volume per wiki:
  • What is the amount/% of searches initiated in the go-bar that end up on the special page?
  • What is the amount/% of users that get redirected to the special search page after doing a go bar search?
  • What amount/percentage of queries that get redirected to special:search had no autocomplete suggestions?
  • What amount/percentage of queries that have no autocomplete suggestions also have zero full text search results (i.e. 0 autosuggest suggestions > 0 special:search results)? inverse: what amount/percentage of queries with no autocomplete suggestions do have results in special:search?
• Are users clicking through on special:search results, especially when they are redirected there and there are no autocomplete suggestions?
• Search engagement: TBD

Languages of interest
We are interested in the following emerging languages for the search experimentations:

Priority 1:

• Arabic
• Bengali*
• Spanish
• Portuguese*
• Russian

Priority 2:

• French*
• Korean*
• Indonesian
• Ukrainian
• Thai*
• Malaysian (?)
• Hindi
• Tagalog
• Afrikaans
• Cantonese
• Malayalam
• Telugu

*These languages are part of wikis to avoid (below)

Wikis to avoid:

Note that we want to avoid/be mindful when doing analysis for wikipedias that have new vector deployed on desktop for the go bar, as that might affect our metrics.
List of wikis of early adopters for new Vector skin (and go bar improvements): https://www.mediawiki.org/wiki/Reading/Web/Desktop_Improvements#List_of_early_adopter_wikis_(test_wikis)

Dashboard:

• Dashboard on Superset with monthly updated data with notes from notebook.
• Historial metrics are stored in a hive table.

Supporting information

• Search team has a script in R to compute search traffic metrics on specific wikipedias
• We should be able to get historical data for the last 90 days instead of waiting for 2 weeks thanks to the new logging infrastructure