As part of the work on MinT for Wiki Readers MVP (T359072), we want to learn about its impact.
In addition to impact metrics (T373862) we want to consider direct feedback from users. This ticket proposes to show a simple feedback survey when users access the translation view (T359801).
The goal is to check with users how useful they find the machine translation provided. Thus, we may want to ask users to indicate the level of usefulness, as well as other factors that help put that in context. For example, we can consider asking whether their evaluation is based on this particular instance or their general use, including this and previous machine translated articles.
As part of this ticket, we'll define the specific questions to ask, and design the way to invite users to take the survey without getting in the way.
Proposal
A simple in-context survey to provide users with one single question and a follow-up one in case they answer the first one.
Although Quick Survey extension cannot be used in this context (does not work on Special pages), the solution proposed is aligned with it and reuses aspects such as the privacy statement language.
Try prototype or explore in Figma.
Goals:
- Clarity. We want to ask users about usefulness of the feature in a way that leaves no ambiguity.
- Simplicity. We want to be simple to build and use. We want the prompt to be compact and the survey to be quick to fill. Users came for the content and we are asking the to do something different. In this scenario, if it is not a very simple task users may just skip it.
Key ideas:
- Show the invite for feedback at the end of the page. It makes sense to ask for the evaluation of the content after the user had a chance to look at it. Given that sections are collapsed on the translation view, the request for feedback should be visible when users skim the translated contents.
- Ask a simple yes/no question, and show an optional follow-up question for extra context. In order to balance being compact with getting useful information, the approach provides a simple yes/no question about the usefulness of the translated contents. After selecting an answer, a follow-up question is shown for users to select which aspects they found useful or not useful.
- Additional aspects are intended to check specific value propositions:
- Enabling to access contents that would be harder for users to access in their language (or not).
- Translation quality limiting the learning (or not).
- Usefulness as a way to get the gist of a topic (or not)
- Effect of technical aspects such as loading time.
- A final confirmation is shown with a link to the project talk page allowing users to provide more detailed feedback (opening in a new window/tab to avoid losing context)
- We may want to coordinate with communities to make sure the UI messages are localized before the survey is exposed to users. In this way, monolingual users will be asked in their language.
Technical aspects:
- The instrumentation infrastructure can be considered as a lightweight way to capture the results (among other options).
