As part of the work on MinT for Wiki Readers MVP (T359072), we want to learn about its impact.
This ticket proposes to analyze the available data to identify signs that show both its usefulness to users and any potential problematic consequences based on the data we capture from the 23 pilot Wikipedias.
[An initial analysis](https://nbviewer.org/urls/gitlab.wikimedia.org/kcvelaga/lpl-fy25/-/raw/main/082024_mint_spike.ipynb/%3Fref_type%3Dheads) after a high peak of activity was produced when the entry point at the article footer (T363338) was made available (disabling it later to avoid servers going over capacity). We want to make sure that, the article footer entry point has been restored before completing the analysis in the current ticket.
As part of this ticket, we may want to explore some relevant questions:
- **User interest.** How many users access machine-translated articles, and which percentage of the mobile web traffic for their wiki do they represent (our initial target was to reach 3%).
- **Negative effect in other reading activities.** Does reading translations result in reading fewer articles on the wiki?
- **Negative effect in other editing activities.** Does the availability of translations result in less editing activity to create articles in the local wiki?
- **Funnel overview.** How is their usage pattern. Do they spend time reading several sections of the article, do they navigate to other articles or translations? How much time do users spend in a machine translated article, and how this compares to the time spent in other articles?
- **Content discovery.** Which is the difference of coverage between the translation consumed and the local article available (e.g., MT may be used more on articles that exist but are less than 5 paragraphs, compared to missing articles or longer ones)? Do people select different source languages based on their content available? Which kind of topics do users read as machine translations, which coverage provides the wiki for those, and how do those compare to non-translated articles?
- **Appeal to translators.** Is this a feature only useful for readers, or is it also encouraging the creation of translations? How is the availability of an MT option affecting the [funnel of the mobile translaiton funnel](https://htmlpreview.github.io/?https://github.com/wikimedia-research/content-translation-funnel-analysis/blob/main/T328913_sx_mobile_entry_points.html) (e.g., is the availability of the MT option reducing the number of people that access Content Translation and leaves it without translating because they were actually looking for MT to read)?
**More context about the annual plan target:**
This area of work is in support of [the Wikimedia Foundation Product & Technology Objective and Key Results (OKRs) ](https://meta.wikimedia.org/wiki/Wikimedia_Foundation_Annual_Plan/2023-2024/Draft/Product_%26_Technology/OKRs) of the 2023-24 period. Although the MVP was launched at the end of such period, the idea was to leave time for users to use the feature before assessing the impact. In particular, the target was defined for the key result WE2.2 in the following hypothesis:
> **Scaling Open Translation service will increase page interactions from underserved communities**
> Exposing our machine translation service to support the automatic translation of Wikipedia article contents. Make the service available for 10 languages not supported by commercial vendors such as Google Translate. This will increase page interactions from underserved communities by making millions of Wikipedia articles more accessible with automatic translation.
> Success will be measured after 6 months by a 3% increase in page interactions on the pilot wikis driven by the views to automatic translations of content.
____
This ticket is focused on measurements. Related tickets enable direct user input:
- {T381886}
- {T364301}