iOS Tabs - Usability test analysis (Protocol)

Participants

Completed all parts of the test:

1. M, Female, 30, Japanese (Japan)
2. S, Female, 30, Japanese (Japan)
3. A, Female, 40, Japanese (Japan)
4. M, Female, 33, Japanese (Japan)
5. S, Female, 22, Arabic (UAB)
6. N, Female, 34. Arabic (Saudi Arabia)
7. T, Male, 28, Arabic (Egypt)
8. J, Female, 39, English (Philippines)
9. J, Female, 28, English (Malaysia)
10. M, Female, 51, English (Australia)
11. N, Female, 29, English (Australia)
12. R, Female, 36, English (Indonesia)
13. J, Male, 30, English (Philippines)

Completed only first part of the test:

14. A, Male, 34, Arabic (Saudi Arabia)
15. I, Male, 20, English (Malaysia)

Test remarks

• After setting up iPad screener questions but still ending up with iPhone users, I contacted userlytics support. Their answer: iPad is currently not supported by the platform. Their answers were considered in the analysis (3 more iPhone users).
• Two participants did not complete the second part of the test (prototype). That’s why questions 1-7 feature 15 participants and questions 8-18 feature 13.
• For this usability test, the definition of a significant score for a task is more than 50%. Reasoning: the test is unmoderated, tab preferences are subjective, and the prototype is basic.
• I observed two concepts of how people open tabs: Using search or by long pressing links in articles.

ℹ️ Recommendations

• Keep the current back behavior that takes users back across tabs as a tab-agnostic back behavior has only been mentioned by 1/13
• Always show the label of the previous destination next to the back button to make navigating articles easier. It has helped users immensely to navigate the prototype and understand what happens. It is recommended that the destination is always displayed on the back label. If the label is too long, apply truncation.
• Introduce interface feedback (e.g. animation) when a new tab opens on the tabs icon so users realize that a new tab has opened (similar to Wikipedia on Android). Some users didn’t realize the tab opened in the background since the interface didn’t respond.
• Related: Provide two options after the user’s long press: "Open in new tab" and "Open in new background tab"
  • Change the default tab behavior ("Open in new tab") to open the new tab and navigate to it.
  • Some participants expected to navigate to the tab after tapping the link.
  • Alternative: Introduce foreground/background tab setting (not recommended) as a user’s preference could vary per click.
• Onboarding: show a tooltip for both the tabs overview and opening new tabs (use two contextual onboarding tooltips, with a "next" button and, e.g. 1/2 and 2/2 indicators)
• For a future version, this could be considered:
  • Support gestures to move between tabs
  • Auto-close setting for tabs
  • Consider adding a link to reading history in the tabs overview more menu as it’s related

Summary

1. Imagine you’re curious about something and decide to look it up on Wikipedia. As you read, you find related articles that interest you and want to read them without losing your place. The first part of the usability test consists of questions about images of potential designs. In the second part, you will explore a very basic prototype of the new tabs feature. The design is not final, and some parts are simplified, missing, or buggy. In the tabs overview, you will only see the titles of the articles — there are no visual previews yet.

No recommendations, as these are instructions only.

2. If you wanted to check if you have any other articles open, what would you do on this screen?

ℹ️ Recommendation: Most participants identified the tabs icon at the top right. Introduce onboarding measures to make sure everyone is aware of tabs.

• 8/15 would click the tabs icon at the top right
• 2/15 didn’t answer the question
• 2/15 would click "Save" at the bottom
• 1/15 would use gestures (swipe up)
• 1/15 would click the account icon
• 1/15 would use the search

3. If you were reading an article and wanted to return to a page you visited earlier within the same tab, how would you do that?

ℹ️ Recommendation: Excellent score. The top left back button is an established pattern to navigate to the previous article. No action is needed.

• 11/15 would tap back at the top left
• 2/15 would tap the tabs icon at the top right
• 1/15 would tap the table of contents icon at the bottom left
• 1/15 would tap the history tab in the app’s main navigation

4. How would you feel about the app automatically closing open tabs with articles after a certain amount of time, like after a day, a week, or a month?

ℹ️ //Recommendation://Most participants dislike the idea of auto-closing tabs. A setting to auto-close could be considered in a future version but is unnecessary for V1.

• 9/15 dislike the idea of tabs automatically closing
• 6/15 like the idea of automatically closing it (e.g. a week, month or within a customized period)

5. If you searched for an article and it was already open in a tab, what would you expect to happen if you tapped on it?

ℹ️ Recommendation: To reduce tab overload, we should direct users to the already open tab.

• 10/15 expect that tapping takes you to the article that is already open in a tab
• 5/15 answers were formulated vague/unclear (e.g. "I expect to see the Wikipedia's certain page", "I expect it to display accurately", "Select", "View it in History", "It will give me the main article")

6. Imagine you have the Wikipedia app installed on your phone and searching for Polar bear using Google Search in your web browser, like Safari or Chrome. You see a link to the Wikipedia article for Polar bear and you tap on it. What would you expect to happen next?

ℹ️ Recommendation: Majority expect it to open in the app as default. No action is needed.

• 8/15 mention that it would open in the app
• 5/15 mention to see the Polar bear article
• 1/15 mentions "same subject"
• 1/15 mentions "it gives me all the information related to the topic"

7. When you tap the Wikipedia link for "Polar Bear" on Google, do you expect it to open in a new tab or replace the current one in the Wikipedia app?

ℹ️ Recommendation: Majority expects that it will open a new tab instead of the current one. This should be aligned with the behavior from 5 (if a tab is open with the article, take users to it).

• 10/15 expect it to open in a new tab
• 4/15 expect it to open in the current tab
• 1/15 answered with "Yes"

8. We’re done with the first part of the questions with static images. In this second part, you’ll explore the very basic prototype of the tabs feature. Please install the app prototype on TestFlight

https://testflight.apple.com/join/7WPD9MCZ\

Please think out loud as you go. There are no right or wrong answers — just explore as you naturally would.

No recommendations, as these are instructions only.

9. In the Wikipedia app prototype, search for "Dog" and open the article.

ℹ️ Recommendation: The prominent new search input field at the top of the article works perfectly.

• 13/13 participants have no issues opening the dog article

10. While reading the Dog article, find a link to another article that interests you. How would you open that link in a new tab?

ℹ️ Recommendation: Most participants knew how to open a new tab. It is recommended, though, to have an interface respond to convey that a new tab has opened (animation).

• 7/13 have no issues opening a link in a new tab by long-pressing
  • RS: Lack of interface response (e.g. highlight) that the tab opened confuses some
• 2/13 use search to open articles in a new tab (by long pressing a search result)
• 4/13 don’t open the article in a new tab (misread instructions)

11. Follow a few more links to new articles if they interest you. Open some of them in a new tab and some in the same. Stop when you feel ready.

ℹ️ Recommendation: No actions needed, as 12/13 knew how to open a new tab after being prompted again.

• 10/13 long press a blue link intuitively to open other new tabs
• 2/13 use search to navigate to other articles and to open new tabs
• 1/13 doesn’t open links in new tabs (long press not discoverable)

12. If you haven’t done so, you can long press links in an article and open it in a new tab.

No recommendations, as these are instructions only.

13. Switch between your open tabs to check out the articles you’ve opened.

ℹ️ Recommendation: Tab overview button at the top right works. An additional long press option to open the tab in the foreground is recommended. Consider swipe actions in a future version to make tab navigation easier (not needed for V1).

• 9/13 have no problem switching tabs and checking out articles they opened
  • "Can I swipe up?"
• 3/13 expect that tabs open in the foreground. They didn’t realize that tabs were open in the background.
  • "It doesn’t work!"

14. After switching tabs, where would you expect the back button at the top to take you? Please precisely describe your thoughts out loud.

ℹ️ Recommendation: Always show the destination label at the top left next to the back arrow. Do not use "Back" as the label. If the label’s too long, truncate it with an ellipsis. Keep the current back behavior (across tabs), as only 1/13 mentioned that the back button should be tab agnostic.

• 6/13 expect that it takes you back across tabs
  • "Takes me to the article I have seen before"
  • "I expect it goes to the last page I opened"
  • RS: Precise back helps a lot
• 3/13 provide an imprecise expectation
• 1/13 expects that the back button is taking you to the previous article in the same tab (agnostic)
• 1/13 has no idea where the back button takes them
  • "Maybe the home page"
• 1/13 describes what the back button does and understands the concepts (didn’t have any specific expectations)

15. Press the back button now and describe what happens. Is this where you expected to go? Please share your thoughts out loud.

ℹ️ Recommendation: Most folks are satisfied with the prototype’s back behavior. Tab agnostic back behavior was mentioned only by 1/13.

• 8/13 mention "yes" it’s where they expected. It takes you back to the article they read before
• 2/13 provide an imprecise expectation
• 1/13 mention "no" it wasn’t where they expected. It goes back to the first article (Dog) but didn’t, so it was unexpected.
• 1/13 expects that the back button is taking you to the previous article in the same tab (agnostic)
• 1/13 have no idea where it would take them, but after realizing what it does (going back across tabs) it made sense to them
  • "I guess that’s a normal reaction."

16. Close one of the tabs you no longer need.

ℹ️ Recommendation: Perfect score, no actions needed.

• 13/13 have no problems closing a tab

17.

Answer this question only if you're using an iPad. Skip it if you're on an iPhone.

Switch to the "Explore" main navigation item, tap on an article in the Explore feed, and click a few links to that article.

After that, switch to "History" main navigation item, tap the tabs icon at the top right and switch to the article you last visited from in Explore.

After these two steps, what do you expect to happen if you tap the "Explore" main navigation item again now?

18. How did it feel to browse Wikipedia with tabs? What worked well, what was confusing, and is there anything you’d expect based on how you use tabs in Safari or Chrome?

ℹ️ Recommendation: Most participants like the feature as is. Consider onboarding measures,

• 9/13 like the feature
  • "It’s very smooth but I can't switch to the new tab I opened"
  • "In Safari it is natural to have tabs but didn’t expect it in the Wikipedia app. I want to know that it’s possible to open articles in tabs"
  • "It’s very similar to Safari or Chrome, but I prefer being taken directly to the tab"
  • "This is so easy and useful. This was a great experience."
  • "Very fast and simple. Expects guidance when using it for the first time."
  • "Very useful when you search for various subjects. It’s like your history."
  • "It was ok; it reminded me of Safari and Chrome."
  • "It’s excellent, I wouldn’t like if it automatically closes tabs!"
  • "I expect an option to go to my reading history from here"
• 2/13 provided an unclear answer
• 1/13 mention thateverything gets more confusing (I’ll use the browser)
• 1/13 mention that it was ok but still prefers using Wikipedia in the browser
  • "It was not especially convenient."