As part of the success metrics identified for Content Translation version 2, we want the success rate for new editors using the tool to increase.
We want to measure the percentage of translations by (new) editors that are completed successfully.
- We want to have separate measurements for (a) "new editors", and (b) "existing editors" as a reference.
- We want to have separate measurements for activity from (a) all versions, and (b) version 2 of the tool. That will allow to understand the overall impact as version 2 becomes the default but also to "zoom in" to focus only on version 2 (where the improvements are made) when needed.
- Success levels: We want to capture the translations that resulted in a published article (success), those that were not published because an error prevented from publishing, and those where users just abandoned the process and leave the translation incomplete for the given time period (i.e. the user started the translation but didn't pressed "publish").
- New editors are defined as users that created their account during the last 6 months (the initial criteria user for new editor experiences research).
- Time period: We want to group the results in 30 day periods. Analysing the translations that were started in a given month. We may need to consider a delay to give enough time to be sure that "abandoned" translations are not misclassified.
An example representation is shown below, where the percentage of successfully published translations increases over time, as errors and abandoned translations decrease. The graph shows, regardless the number of new users or the translations they make, how likely they are to succeed or face problems.