Page MenuHomePhabricator

Collect data on all key Content Translation user flows
Open, HighPublic

Description

Users move through different paths when using content translation, and we want to have a better understanding about how many users take each path and the results they produce.

Translators can enter the tool through different entry points (Contributions page/menu, interlanguage links, etc.), start a translation in different ways (searching for a specific article, picking a suggestion, or selecting an article they kept for later), and advance through different stages for the translation (starting a new translation, continuing an existing one, and publishing). Finally, the published content may survive, be edited further, or be deleted by the community.

Important questions this data could help answer include:

  • How much publication can be attributed to each entry point?
  • Are users coming from campaigns more or less successful than other users?
  • How many translations face publish failures, and how many eventually overcome them?
  • Which type of suggestions are most effective, not just in getting people to start, but also in getting things published?
  • With AX, how often do people translate in multiple sessions versus a single one?
  • On mobile, how often do users' sessions stop without passing through the confirmation dialog (suggesting that they lost their in-progress work)?

Related Objects

Event Timeline

Pginer-WMF renamed this task from Measure Content Translation funnel to Visualize Content Translation funnel to better understand different translator paths.Aug 27 2019, 1:24 PM
Pginer-WMF added a project: CX-boost.

We should have metrics in place to understand the user workflow when it comes to the use of multi-publish:

  • User loads draft directly - by the use of a bookmark the user might publish a partial translation and then go directly to the same translation to complete it
  • User uses the published section of the dashboard - metrics for clicks on Published plus metrics on continue translation button
  • User starts a new translation of an existing article - metrics for overrides of published articles for the same user
  • User starts a new translation of an existing article - metrics for overrides of published articles for a different user

Another relevant aspect to capture could be the time it takes to complete the steps (in time or number of sessions). So the measurement proposed for T243387 can be also integrated here.

Pginer-WMF raised the priority of this task from Medium to High.Apr 16 2020, 8:35 AM

This is a pretty awesome diagram! I know all of these points, but I didn't realize there are so many of them.

I've transformed @Pginer-WMF's workflow diagram into one showing the specific events we plan to use, and how they will relate to each other. The immediate task is to review this to make the events capture all the key paths we want to understand. After that, I will move on to T263853 (which will invove updating the diagram with any new events).

Source file:

Thanks, @nshahquinn-wmf. This looks good and it is easy to follow which user action is represented in each step.
Some comments based on the different ways to start a translation:

  • Distinguish direct links. Regarding direct links ("dashboard open > direct") We may want to distinguish links to (a) the dashboard in general and (b) those pointing to a specific article to translate. The later require to just confirm the dialog to get the translation started and represent a more specific intent.
  • Campaigns. Related to the above, in some cases the links include a campaign identifier that is useful to identify which activity originate as part of a given external tool or event. This is something we may want to analyze later (e.g., comparing the success ratio for translations coming from GapFinder vs. those using the suggestions integrated in the dashboard)
  • Grey interlanguage links. Currently the list of interlanguage links includes grey links to surface that content is missing in languages you may know. This is used as an entry point to start the translation of such specific article into the selected language where it was missing. As part of the desktop refresh project this entry point will change (won't be presented as grey links) but we will still be supporting the usecase of starting a new translation from the language selection process in the article. so we may want to consider the event but with a more generic name not referring specifically to grey links.
  • Invite when creating a new article. When a new article is created, an invite is shown for the user to consider creating the article as a translation (example). The invite could be generic (to let people know about CX, leading to the dashboard and shown only once) or specific (a particular page was identified as the article the user was trying to create and is suggested to translate). More details in T232372.
nshahquinn-wmf renamed this task from Visualize Content Translation funnel to better understand different translator paths to Instrument the full Content Translation funnel.Oct 26 2020, 2:05 PM
nshahquinn-wmf renamed this task from Instrument the full Content Translation funnel to Instrument all key Content Translation user flows.
nshahquinn-wmf renamed this task from Instrument all key Content Translation user flows to Collect data on all key Content Translation user flows.May 5 2021, 12:30 PM