In T187827, @MNeisler replicate what I (@chelsyx) did in T177534 to compute some search metrics on Wikimedia Commons, in order to get familiar with the data analysis tools we are using (see the epic ticket T185363 for more details). She found that the desktop full-text search-wise CTR is only 3.17% on Commons in February 2018 (see her report for more details), which drops a lot comparing to the first analysis in November 2017 (10.42%).
Potential direction for the investigation:
- Get the search-wise CTR from Nov to now. Is it a gradual decline or sudden drop?
- Check the number of search events and sessions, and then compute other versions of CTR (session-wise CTR, total clicks / total impressions). Do we see the same pattern?
- Before and after the drop, does the distribution of the users (in terms of browser and operating systems) change a lot?
- Check the log of several sessions with no click, is there any pattern?
- Is this happening on other wiki as well?