Create or revive a dashboard for the Search Platform team (formerly Discovery) that includes the below metrics, and the question we are trying to answer with them:
- Be able to filter by the following dimensions: platform (Desktop, Mobile Web, Android, iOS); bot vs non-bot, language
- What do included metrics look like when we segment users/queries?
- Search Engagement (= number and percentage of queries with a dwell time > 10 seconds) [note to self: Search Engagement formerly known as Search Satisfaction/User Engagement].
- Are users finding full text search results relevant/useful?
- Number and percentage of full-text searches, "Go" box, morelike, autocomplete searches
- What type of searches are being used?
- Number of WDQS time outs
- How often is Wikidata Query Service failing to return results for a user's query?
- Number and percentage of queries with "did you mean" suggestions.
- How well are we accommodating imprecise search queries?
- Number and percentage of "did you mean" suggestions clicked on.
- Are relevant are our results for imprecise search queries?
- Number and percentage of abandoned sessions.
- How many users are unsatisfied with search results that they are leaving?
- Number of requests to WDQS and Linked Data Fragments, dumps, mediawiki APIs
- What services are people using to get data from Wikidata?
- Number and percentage of zero results
- What happened (recently) that may have drastically affected how many queries are returning zero results?
- Top queries and top keywords
- Are there any common query patterns worth doing anything about?
- Top returned documents (articles) and top clicked through documents
- Are there patterns in specific search results that are worth doing anything about?
We have excluded any metrics that would require building a new thing: i.e. a "smiley face" search satisfaction survey that would need to be built.
Search Platform currently lacks analyst support in two major ways:
- Technical. Legacy bespoke dashboards were built in shiny and R and the team lacks the current resources and technical expertise to maintain this code ourselves.
- Data expertise. The team is able to do rudimentary data analysis, but lacks the expertise to really validate statistical significance, whether we are capturing the right signals to test our hypotheses, etc.
Even if a dashboard were to be built for us, it runs the risk of growing stale and obsolete without the ability to actively maintain/tune it, which we are unable to do on our own. To ensure long-term value and avoid wasted effort in constantly (re)building dashboards that grow stale, it would be ideal to have access to resources to help us maintain our metrics in the long term.
Include relevant timelines/deadlines, OKRs
we would like to have this in time for the next quarterly planning cycle
How will you use this data product?
To understand what Search performance/features/functionality looks like in production, and discover and prioritize future Search Platform team work.
Is this request urgent or time sensitive?