The Content Translation tool now has custom suggestions menu and various ways for users of CX to customize suggestions, which was developed as a part of T113257.
An initial analysis has been done as part of T381391: Visualize use of suggestions over time for mobile translations.
The team would like to monitor the use of the custom suggestions over time and relevant user interactions for the menu. The goal is to add relevant aggregate metrics to CX key metrics Superset dashboard.
Steps involved:
- Propose metrics to be monitored
- Finalize metrics to be monitored
- Assess if an aggregation data pipeline is needed
- Develop queries for aggregation
- Develop Airflow pipeline for aggregation (if needed)
- Addition of metrics to the dashboard
A new tab has been added to the key metrics dashboard: https://superset.wikimedia.org/superset/dashboard/p/opjO9Nyv5Ry/
Proposed metrics (for monitoring)
- Daily unique users interacting with the custom suggestions menu
- Daily clicks to open the custom suggestions menu
- Aggregate of high level of sources for filters selected (collections topic area, popular articles, previous edits)
- Aggregate of specific sources filters selected (for example, names of the topic areas, or collections)
- Proportion of users that proceeded to translation start step by selecting a suggestion from the custom suggestions
to be considered later for funnel analysis
- Proportion of users (of the ones who opened the menu)
- that selected at least one filter within the custom suggestion menu
- that confirmed with at least one filter
- that closed without any confirmation
- Proportion of users that successfully published a translation after selecting a suggestion from the custom suggestions