Background

Two different designs are in contention for updates to make to the Android app and article navigation experience.
The following outlines the research plan to assess and compare the efficacy of both concepts in improving the app such that there is (a) greater focus on articles as a primary destination; and (b) improving the experience when reading and taking actions in articles.

Concept 1. Floating Article Queue – add a floating element that quickly opens the articles stack when it is minimized. See details in T195335
Concept 2. Explicit Reading Mode – add 'Reading' and 'Search' as explicit bottom navigation menu items. See details in T196372

Goals of the study

Assess two different proposals for improving navigation and article reading experience in the Android app and compare their efficacy against the current app experience.

1. Overall conception – how does the new design compare with their existing impression of the current Android app?
2. Usability of the new feature, particularly:
   1. Article reading experience
   2. Access and browsing other parts of the app
   3. Searching and finding articles
   4. Customization (access to changing the feed, etc)
3. Better understand how users are navigating through the app from general browsing to reading articles

Test format

Part 1. Diary study
Participants are provided with a prototype A or B alpha build and asked to use the app as they normally would, for 7-10 days. Users will be prompted to provide regular feedback about their experience with the app.

Part 2. Remote moderated usability session
This session will be conducted after the diary study with the same participants. It will capture participants’ usage of the tool to complete core tasks more naturally after they have been exposed to the app, as well as getting feedback on their overall experience.

Participants

Testing will be conducted with existing regular users of the Wikipedia Android app.
Ideally there will be 6 participants for the study for each prototype being tested.