This task encompasses investigation into:
- Whether search metrics we are tracking are getting the correct data.
- Whether we are getting the proper data from Desktop and Mobile Web, and Android and iOS Apps
-----
== Issues and anomalies ==
These are the issues and anomalies with [our dashboard](https://discovery.wmflabs.org/metrics/) that's been found so far. Some can be fixed immediately. For those that need further investigation, we will create new tickets.
**Infrastructure**
[] All instances in the shiny-r project needs to upgrade as soon as possible (T204688, T204505)
**Bugs in dashboard**
[x] On [SRP visit time](https://discovery.wmflabs.org/metrics/#spr_surv), the results for “How long Wikipedia searchers stay on the search result pages” looks broken because the results for the three groups (English, French and Catalan, and Other) are all identical
[x] [Dwell time on visited page](https://discovery.wmflabs.org/metrics/#survival) stop updating since 2017-8-27
[x] On [metrics summary](https://discovery.wmflabs.org/metrics/#summary), adding explanation of user engagement calculation could be helpful. Same on [KPI: User engagement](https://discovery.wmflabs.org/metrics/#kpi_augmented_clickthroughs)
[] On [KPI: User engagement](https://discovery.wmflabs.org/metrics/#kpi_augmented_clickthroughs), make sure the calculation is correct. Also adding user engagement graph by platform may be helpful, since CTR on various platforms seem very different
[x] On [desktop events](https://discovery.wmflabs.org/metrics/#desktop_events), adding CTR graph may be helpful
[] On [PaulScore Approximations](https://discovery.wmflabs.org/metrics/#paulscore_approx), paulscore for autocomplete searches is 0
[] On [mobile app events](https://discovery.wmflabs.org/metrics/#app_events), Android events count drop since 4/20, iOS events count drop since 6/19. Their [load time](https://discovery.wmflabs.org/metrics/#app_load) also increase around the same time. These should relate to app eventlogging schema changes and need to be fixed and documented
[] [Invoke Source on Mobile App](https://discovery.wmflabs.org/metrics/#app_invoke_source) stop updating since 2017-10-4
[x] On [API calls by referrer class](https://discovery.wmflabs.org/metrics/#referer_breakdown), need to add the annotation about UDF change in 2017-06-29
**Anomalies in metrics** (may need further investigation)
[] On [mobile web events](https://discovery.wmflabs.org/metrics/#mobile_events), there was a dip in events count from 8/15 to 9/5
[] On [full-text search via API](https://discovery.wmflabs.org/metrics/#fulltext_search), direct usage increase since 3/22 -- looks like a bot (click to include bot in the [fulltext ZRR](https://discovery.wmflabs.org/metrics/#failure_suggestions)). External usage increase since 8/12, and the [fulltext ZRR](https://discovery.wmflabs.org/metrics/#failure_suggestions) also increase at the same time
[] Something seems to be changed with the identification of referrer class -- looks like direct referred traffic are re-categorized as internal traffic since 4/1 on [morelike search](https://discovery.wmflabs.org/metrics/#morelike_search) and [prefix search](https://discovery.wmflabs.org/metrics/#prefix_search)
[] [Geographic search](https://discovery.wmflabs.org/metrics/#geo_search) increase hugely since 8/21, its [ZRR](https://discovery.wmflabs.org/metrics/#failure_breakdown) and the [overall ZRR](https://discovery.wmflabs.org/metrics/#failure_rate) also increase
[] [Language search](https://discovery.wmflabs.org/metrics/#language_search) increase hugely since 3/1
[] On [sister search traffic](https://discovery.wmflabs.org/metrics/#sister_search_traffic), mobile web traffic drop since 6/29