# Usability test analysis and recommendations
This is the analysis of the [Android Add image AI trainer MVP: Research](https://docs.google.com/document/d/1Pf_uFTA2kW-B2zzrYUHd4u_DDnrDmY55QBNXqb8mnAI/edit#). This is an independent analysis, performed by the Android team and serves as a basis for discussion with the Growth team. The Growth team performed an [additional analysis](https://docs.google.com/spreadsheets/d/1AI3UKN5Le969QCc34jWQ9JLIWwr5ub9LfpE8MU48SF0/edit#gid=0) of the usability tests on [usertesting.com](https://app.usertesting.com/dashboard#!/study/3429612/sessions).
---
## 🚨 **WORK IN PROGRESS** 🚧
- [x] Rank usefulness of information (rating scale)
- [x] Analyze tasks / identify patterns
- [x] Formulate / define action items
- [x] Process follow-up questions (RS)
- [x] Process follow-up questions (JT, RS, RH, MM)
- [x] Review recommendations (JT)
- [ ] Prioritize recommendations and create Phab tasks (JT, RS)
- [ ] Transfer findings to Mediawiki page (JT)
---
## TOC
1. TL;DR — Recommendations
2. Analysis of tasks
3. Analysis of rating scale
4. Analysis of follow-up questions
5. Appendix
---
## 1. TL;DR — Recommendations
1. Optimize designs for smaller screens, as some participants had display issues on narrow viewports or with an increased text size setting on their device (accessibility feature). These are the suggestions based on the observations in the test:
- The bottom sheet for image suggestions needs to be draggable in order to reveal the article contents below it. Also, participants tried to interact with the handle bar at the top of the bottom sheet. `#needsDevelopment` `#mediumFeasibility`
- Also worth considering a max height of the bottom sheet in order to not cover the article completely. `#needsDesign` `#lowFeasibility`
- Optimize tooltip positioning and handling on smaller screens, as they are cut off on smaller screens. `#needsDevelopment` `#mediumFeasibility`
2. Optimize design for different themes. The tests revealed the following issues:
- Create more suitable 'Train image algorithm' onboarding illustrations for all different themes. `#needsDesign` `#highFeasibility`
- The checkbox items in the the 'No' and 'Not sure' dialogs have issues in dark/black theme and need to be optimized. `#needsDesign` `#highFeasibility`
- The element of positive reinforcement/counter has displays in the dark/black theme and needs to be optimized. `#needsDesign` `#highFeasibility`
3. Provide an easy way to access the entire article from the feed, e.g. by incorporating a 'Read more' link, tappable article title or showing the entire article right from the beginning. `#needsDesign` `#highFeasibility`
4. Show success screen (see [designs on Zeplin](https://zpl.io/2vE54Bv)) when users complete daily goal (8/8 image suggestions) `#needsDevelopment` `#highFeasibility`
5. Optimize the 'No' and 'Not sure' dialog copy to reflect that multiple options can be selected. Some participants weren’t aware that multiple reasons can be selected. `#needsCopywriting` `#highFeasibility`
6. Optimize copy of the 'opt-in' onboarding screen, as there’s an unnecessary word at the moment ('We would ~~you~~ like (...)'). `#needsCopywriting` `#highFeasibility`
7. Suppress “Sync reading list” dialog within Suggested edits as it’s distracting from the task at hand. `#needsDevelopment` `#highFeasibility`
8. Show full image in the feed instead of a cropped version, as some participants had issues identifying the subject of the image because of a suboptimal crop (e.g. camera or rodent) #needsDesign #needsDevelopment #mediumFeasibility
9. Incorporate pinch to zoom functionality, as participants tried to zoom the image directly from the image suggestions feed. `#needsDevelopment` `#lowFeasibility`
10. Incorporate gesture to swipe back and forth between image suggestions in the feed, as participants were intuitively applying the gestures. Per previous conversation with @Dbrant, there are questions that need to be answered from a technical perspective. `#needsDevelopment` `#mediumFeasibility`
- //DB: Suppose the user swipes forward accidentally, and then immediately swipes back. Should that count as a 'Not sure' event?//
- //RS: It should not count in order to not motivate people to do it.//
- //DB: How do we know if the swipe was accidental or deliberate?//
- //RS: Would it be possible to set a delay before counting it?//
-//JT: Explore the option of undo but you can only go back once. I'd prefer people not swipe forward.
11. Optimizations for 'Suggestion reason' meta information
- Optimize copy 'Suggestion reason' meta information as the current copy ('Found in the following Wiki: trwiki') is not clear enough. `#needsDevelopment` `#needsCopywriting` `#highFeasibility`
- Might be worth to explore making the 'Suggestion reason' more prominent as participants rated its usefulness the lowest (likely due to low discoverability) `#needsDesign` `#mediumFeasibility`
12. Optimize the discoverability of 'info' button at the top right as 2/5 participants had issues finding it. `#needsDesign` `#mediumFeasibility`
- //JT: What about a question mark instead of an i?//
- //RS: Changing the icon won’t not solve the discoverability issue, I will consider adding an additional label or different positioning of the FAQ.//
13. Optimize design of positive reinforcement element/counter on the Suggested edits home screen, as it was positioned too close the task’s title. `#needsDesign` `#highFeasibility`
14. Remove full screen overlay when transitioning to next image suggestion. This allows users to orient better and keep context after submitting an answer. `#needsDesign` `#highFeasibility`
- //JT/DB: If we remove the full screen overlay, where will we put the positive reinforcement?//
- //RS: Same position as it is currently, but with less / zero opacity for the overlay//
15. Write FAQ page #needsPM #highFeasibility
16. Explore tooltip "Got it" button #needsDevelopment #mediumFeasibility
---
## 2. Analysis of tasks
```
🥰 = Good — Participant had no issues
😡 = Bad — Participant had issues
🤔 = Not sure if good or bad — Participant might had difficulties understanding the question, did not explicitly interact with it or ignored the task completely
```
| **Participant** | **[Tasks](https://docs.google.com/document/d/1Pf_uFTA2kW-B2zzrYUHd4u_DDnrDmY55QBNXqb8mnAI/edit)** | | | | | | | | | | | | | | | | | | |
| | #6 | #9 | #10 | #11 | #13 | #15 | #16 | #17 | #18 | #19 | #20 | #21 | #22 | #23 | #25 | #26 | #27 | #28 | #30
| [Battybrit](https://app.usertesting.com/v/1f7f73ee-d57f-406a-9935-a8183d603dda?encrypted_video_handle=af7349d7-319a-498d-9db1-d011e664a943&shared_via=link)| 🤔 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰 | 🥰
| [brad.s](https://app.usertesting.com/v/ce4ee78e-1a36-45e1-bdfa-a429bc90973b?encrypted_video_handle=c11c66ce-3629-493f-94ba-40122a9b6a58#!/notes) | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰| 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰
| [147qb](https://app.usertesting.com/v/e2c0c308-2ef1-44ff-b721-87078d615413?encrypted_video_handle=2825e573-b49b-42ea-bc11-21b53375f939) | 🤔 | 🥰 | 🥰 | 😡 | 🥰 | 🤔 | 🤔 | 🥰 | 😡 | 🥰 | 🤔| 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 😡 | 🥰 | 🥰
| [TestMaster888](https://app.usertesting.com/v/08b9637c-08c6-48e1-a136-51e87355bba9?encrypted_video_handle=aca2c43f-081c-411e-bd46-77be1ae8b991) | 🤔 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🤔| 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰
| [Cherry 928](https://app.usertesting.com/v/5d186a06-a292-44da-b9e1-3a5719a24a7f?encrypted_video_handle=7b17d7ef-54c3-4f8d-8b32-eb3c6b2351b8) | 😡 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰
| **Overall evaluation** | 😡 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🥰 | 🤔 | 🥰 | 🥰
#### Onboarding and understanding of Suggested edits
##### `Task #6` Do participants understand the tooltip? 😡
- 2/5 discovered the tooltip but had issues understanding it.
- 2/5 did not see the tooltip since it disappeared too quickly.
- 1/5 discovered and understood the tooltip completely.
##### `Task #9` Can participants explain the difference between tasks? 🥰
- 5/5 were able to explain their understanding of the tasks in a sufficient way.
##### `Task #10` Do participants understand what the 'Train image algorithm' task is all about? 🥰
- 5/5 were able to describe the task in their own words well.
##### `Task #11` What do participants associate with the robot icon? 🥰
- 4/5 associated the robot icon with an algorithm, artificial intelligence (AI) or computer program
- 1/5 didn’t know what it means
#### Train AI task - Onboarding and understanding
##### `Task #13` Do participants understand the two onboarding screens? 🥰
- 4/5 understand both onboarding screens.
- 1/5 wasn’t reacting to the second onboarding screen (opt-in).
##### `Task #15` How do participants interact with onboarding tooltips? 🥰
- 3/5 understand the task due to the tooltips.
- 1/5 mentioned that the tooltips are very helpful to understand the task.
- 1/5 understands the task but did not pay attention to the tooltips.
- 1/5 probably did not see or understand the tooltips.
##### `Task #16` Is the tooltip copy clear enough? How’s the timing and positioning of the tooltips on various devices / screen sizes? 🤔
- 3/5 read and understand the tooltip copy.
- 2/5 did not interact with the tooltips.
- 2/5 had tooltip display issues on a smaller phone.
- 1/5 likes that the tooltip mentions the impact (help readers understand a topic)
##### `Task #17` Do participants know what to do after all these onboarding measures? 🥰
- 5/5 understand what to do now.
#### Train images task
##### `Task #18` Do participants interact with the prototype naturally? 🥰
- 4/5 are mostly comfortable interacting with the UI and make educated decisions.
- 3/5 do not navigate to the file page without being prompted.
- 2/5 navigate between the article and file page intuitively and without issues.
- 1/5 is intimated to make decisions that affect Wikipedia articles, doesn’t know how to interact with the article (//RS: possible due to small screen size//) and doesn’t use file detail page intuitively.
##### `Task #19` Do participants know how to navigate to the file detail page? 🥰
- 5/5 successfully navigated to the file detail page after being prompted.
- 1/5 tapped the 'info i' icon in the feed view first.
##### `Task #20` How helpful is the meta information on the file detail page? 🥰
- 3/5 consider the information on the file page as helpful.
- 2/5 mention that the author is helpful.
- 2/5 mention that the date is helpful.
- 1/5 mentions that licensing info is helpful.
- 1/5 mentions that the image description is helpful.
##### `Task #21` Do participants know how to enlarge / zoom an image? 🥰
- 5/5 tapped the image and used a pinch to zoom gesture to zoom the image.
- 2/5 tried to zoom the image directly from the feed experience.
##### `Task #22` Do participants know how to go back and forth between image suggestions? 🥰
- 5/5 use swipe gestures to navigate back and forth between image suggestions.
- 2/5 tapped the back button at the top left before using the swipe gesture.
- 1/5 tapped the 'info i' button at the top right before using the swipe gesture.
##### `Task #23` Do participants understand the 'Not sure' options? 🥰
- 5/5 understand the 'Not sure' options.
- 3/5 were selecting multiple reasons at once.
##### `Task #25` Do participants understand the 'No' options? 🥰
- 5/5 understand the 'Not sure' options.
##### `Task #26` Do participants scroll or know how to reveal more of the article contents? 🥰
- 4/5 were successful in scrolling the article to reveal more information
- 2/5 wanted to use the pull indicator at the top of the image suggestion to reveal the article below before they scrolled the article
- 2/5 tried to the tap the article title (1/5 scrolled afterwards)
- 1/5 looked for a 'More' button to reveal more of the article’s content, then tapped the 'info i' button at the top right
##### `Task #27` Do participants know how to access the FAQ? 🤔
- 3/5 tap the 'info i' button at the top right to reveal the FAQ.
- 1/5 explained that she would tap the back button and look for an FAQ there (//RS: a possible way to success as there’s an FAQ section in the SE home screen//)
- 1/5 did not notice the 'info i' button at the top right
##### `Task #28` How do participants interpret the element of positive reinforcement? 🥰
- 5/5 understand what it is and identified the element as motivational, encouraging and/or daily goal
- 1/5 wasn’t 100% sure about it but then identified it as a motivational element.
##### `Task #30` Do participants notice the element of positive reinforcement that has been added to the card? 🥰
- 5/5 participants identified the added progress indication in the card
---
## 3. Analysis of rating scale
```
1 = Not at all useful information
5 = Very useful information
```
| **Participant** | **[Rating](https://docs.google.com/document/d/1Pf_uFTA2kW-B2zzrYUHd4u_DDnrDmY55QBNXqb8mnAI/edit)** | | | |
| | First paragraph (#36) | Description (#33) | Filename (#32) | Caption (#34) | Suggestion reason (#35)
| [Battybrit](https://app.usertesting.com/v/1f7f73ee-d57f-406a-9935-a8183d603dda?encrypted_video_handle=af7349d7-319a-498d-9db1-d011e664a943&shared_via=link) | 5 | 5 | 5 | 5 | 4
| [brad.s](https://app.usertesting.com/v/ce4ee78e-1a36-45e1-bdfa-a429bc90973b?encrypted_video_handle=c11c66ce-3629-493f-94ba-40122a9b6a58#!/notes) | 5 | 5 | 4 | 5 | 3
| [147qb](https://app.usertesting.com/v/e2c0c308-2ef1-44ff-b721-87078d615413?encrypted_video_handle=2825e573-b49b-42ea-bc11-21b53375f939) | 5 | 4 | 3 | 4 | 3
| [TestMaster888](https://app.usertesting.com/v/08b9637c-08c6-48e1-a136-51e87355bba9?encrypted_video_handle=aca2c43f-081c-411e-bd46-77be1ae8b991) | 5 | 4 | 3 | 4 | 2
| [Cherry 928](https://app.usertesting.com/v/5d186a06-a292-44da-b9e1-3a5719a24a7f?encrypted_video_handle=7b17d7ef-54c3-4f8d-8b32-eb3c6b2351b8) | 5 | 5 | 5 | 2 | 2
| **Overall rating** | **5** | **4.6** | **4** | **4** | **2.8**
---
## 4. Analysis of follow-up questions
##### 1. How do you think the suggested images for articles are being found? And how would you rate the overall quality of the suggestions?
- 5/5 mentioned that the images presented were relevant.
- 4/5 associated the image suggestions with an algorithm or computer program.
- 2/5 mentioned that the suggestions are associated with keywords.
- 1/5 mentioned these are random suggestions.
##### 2. Was there anything that you found frustrating or confusing, that you would like to change about the way this tool works?
- 3/5 replied that it’s easy to use.
- 1/5 that it’s tedious and cumbersome.
- 1/5 suggested to show more than 1 image choice per article.
##### 3. How easy or hard did you find this task of reviewing whether images suggested were a good match for articles?
- 4/5 find it very easy to evaluate if it’s a good match for the article.
- 1/5 think it’s hard and time consuming but well worth it.
##### 4. Would you be interested in adding images to Wikipedia articles this way? Please explain why or why not.
- 4/5 are interested in such a feature
- 1/5 mentions he would not be interested
- 1/5 mentions that she wants to know how accurate she is when reviewing images
---
## 5. Appendix
Detailed notes and evaluations per participant from usertesting.com.
#### Participant #1: [Battybrit](https://app.usertesting.com/v/1f7f73ee-d57f-406a-9935-a8183d603dda?encrypted_video_handle=af7349d7-319a-498d-9db1-d011e664a943&shared_via=link)
##### Tasks
**Task #1-5 (instructions)**
**Task #6 🤔**
Tooltip disappeared before she was able to interact with it.
**Task #7-8 (instructions)**
**Task #9 🥰**
Understands the rough concept of it.
**Task #10 🥰**
Understands the concept of it.
**Task #11 🥰**
Associates the robot icon with AI.
**Task #12 (instructions)**
**Task #13 🥰**
- Understands first onboarding screen.
- Understands second onboarding screen.
**Task #14 (instructions)**
**Task #15 🥰**
**Task #16 🥰**
**Task #17 🥰**
Understands the tooltips and what to do now.
**Task #18 🥰**
- Comfortable with using the interface
- Does not use meta data intuitively
- Notices the reach the goal message
1. 🥰 Does not see a rodent on the image. Chooses low quality.
2. 😡 Even though an aviator is shown, she chooses not relevant. Realizes that she got it wrong in the next one.
3. 🤔 Not 100% sure if she’s correct, but chooses yes.
4. 🥰 Is confident about the choice.
5. 🥰 Makes informed decisions based on the image.
6. 😡 Does not check meta data.
9. 🥰 Not sure if a person is suited for a surname.
10. 🥰 No, Not enough information
11. 🥰 No, Not relevant
12. 🥰 Not sure, Not enough information
13. 🥰 Yes
14. 🥰 No, Low quality
15. 😡 No, Not enough information
16. 🥰 Not sure, Not enough information
17. 🥰 No, Not enough information
18. 🥰 No, Not relevant
19. 🥰 Not sure, Not enough information (different language)
20. 🥰 Yes
**Task #19 🤔**
- Taps the 'info i' button first.
- Then figures out that she needs to tap the image itself.
**Task #20 🥰**
- Mentions the info about the author, licensing and the date is helpful,
**Task #21 🥰**
Wants to zoom in the image by **pinching** (but it’s not yet implemented)
**Task #22 🥰**
Easily **swipes** to go back to the previous and next suggestions.
**Task #23 🥰**
Explains all options well.
**Task #24 😡 (instructions)**
Didn’t know that she can select more than one reason.
**Task #25 🥰**
Explains all options well.
**Task #26 🤔**
- Would click a button to expand the article (RS: Not sure which button she’s referring to as there is none).
- Then further explains that she would tap the article’s title.
**Task #27 🥰**
Would tap the 'info i' button to reveal more information
**Task #28 🥰**
- Wasn’t 100% sure about it
- Identifies it as a motivational element
**Task #29 (instructions)**
**Task #30 🥰**
Had no issues identifying what’s changed and what the additional number and bar means.
**Task #31 (instructions)**
**Task #32-36 (see rating scale)**
**Task #37 (open format)**
Answer/rating from other people that are using the feature to validate/correct her own choice.
**Final thoughts**
None
##### Follow-up questions
1. I think they are being found by tags on the internet? maybe keywords? I would rate overall quality as good, some obviously were off but that makes sense since many places have similar names or multiple meanings.
2. I didn't find anything confusing but at times I was disappointed with myself because I didn't know if one fit into a yes or no. I like to be accurate and sometimes didn't trust my own judgement.
3. I found some very easy and clearly related or not related but some were tricky because I didn't know the subject well.
4. I would be interested but as mentioned verbally, I'd like to know how accurate I am. I just would not want to be adding wrongly and would like to know my accuracy rate.
---
#### Participant #2: [brad.s](https://app.usertesting.com/v/ce4ee78e-1a36-45e1-bdfa-a429bc90973b?encrypted_video_handle=c11c66ce-3629-493f-94ba-40122a9b6a58)
##### Tasks
**Task #1-5 (instructions)**
**Task #6 🥰**
Understands the tooltip at the bottom.
**Task #7-8 (instructions)**
**Task #9 🥰**
Explains the 'Train image algorithm' task well and knows where images are positioned (on Desktop Wikipedia). Also knows what the others mean.
**Task #10 🥰**
Mostly understands the task.
**Task #11 🥰**
Associates robots with algorithms, even though he thinks that it doesn’t mean anything.
**Task #12 (instructions)**
**Task #13 🥰**
- Understands first onboarding dialog.
- Opts in to displaying his name on the algorithm training page.
**Task #14 (instructions)**
**Task #15 🥰**
Understands the task perfectly due to the tooltip. Says the tooltips are very helpful to understand the task.
**Task #16 🥰**
Likes that the tooltip mentions the impact that his choice has (help readers understand the topic)
**Task #17 🥰**
Mentions tooltips were very helpful. Similar to Google Maps feature (e.g. “does this shop have toilet facilities?”)
**Task #18 🥰**
- Navigates intuitively between article and file details screen
- Doesn’t consider 'Suggestion reason'
- Selects multiple items in 'No' reason
- Selected 'Not sure' only once
1. 🥰 No, Not enough information — doesn’t check for the details of the image though
2. 🥰 Yes, because of Filename
3. 🥰 No, Not enough information — checks out the detail page of the image
4. 😡 Yes (//RS: I don’t think it’s suitable//)
5. 🤔 Yes
6. 🥰 No, Not enough information
7. 🥰 No, Not relevant
8. 🥰 Yes
9. 🥰 No, Not enough information, Don’t know the subject (//RS: Article excerpt’s too short//)
10. 🥰 No, Not relevant, Don’t know this subject
11. 🥰 Yes
12. 🥰 No, Not relevant, Not enough information, Don’t know this subject
13. 🥰 Yes
14. 🥰 No, Other
15. 🥰 No, Not relevant
16. 🥰 No, Not relevant, Other
17. 🥰 Not sure, Not enough information
18. 🥰 Yes
**Task #19 🥰**
Intuitively taps on image card.
**Task #20 🥰**
**Task #21 🥰**
Zooms in the image on the detail page not in the main feed (//RS: There are multiple paths to success//).
**Task #22 🥰**
Easily swipes left and right between image suggestions.
**Task #23 🥰**
Explains all 'Not sure' options well.
**Task #24 (instructions)**
**Task #25 🥰**
Explains all 'No' options well.
**Task #26 🥰**
- Knows how to scroll
- Wants to use the pull indicator
**Task #27 🥰**
Would press the 'info i' icon to reveal more information
**Task #28 🥰**
Identified what the counter is at the bottom (encouragement).
**Task #29 (instructions)**
**Task #30 🥰**
Had no issues identifying wha the counter is.
**Task #31 (instructions)**
**Task #32-36 (see rating scale)**
**Task #37 (open format)**
Nothing to specific (audio broke up in the beginning)
**Final thoughts**
—
##### Follow-up questions
1. using AI. the suggestions were very good and the majority were accurate.
2. N/A
3. very easy and user intuitive
4. yes, as it is so easy and simple to do
---
#### Participant #3: [147qb](https://app.usertesting.com/v/e2c0c308-2ef1-44ff-b721-87078d615413?encrypted_video_handle=2825e573-b49b-42ea-bc11-21b53375f939#!/notes)
- Uses quite a small phone
- Uses accessibility setting that increases the font size in the app
##### Tasks
**Task #1-5 (instructions)**
**Task #6 🤔**
Reads it out loud and funny, not sure if he understood the tooltip though.
**Task #7-8 (instructions)**
**Task #9 🥰**
More or less describes the task’s correctly.
**Task #10 🥰**
Understands the task, doesn’t mention AI though.
**Task #11 😡**
Doesn’t know what it means.
**Task #12 (instructions)**
**Task #13 🥰**
- Understands the first onboarding screen.
- Identifies a copy error “We would ~~you~~ like (...)”
**Task #14 (instructions)**
**Task #15 🤔**
Not sure if he understands completely.
**Task #16 🤔**
Sees the tooltip.
**Task #17 🥰**
Taps 'Yes' accidentally but it seems that he understood the task.
**Task #18 😡**
- Assumes that he needs to know about the article’s topic to make an educated decision if the image fits it or not.
- Is intimated to make decisions for articles in their doctoral thesis's.
- Does not interact with the article
- Does not interact with the file detail page
1. 😡 Not sure, Don’t know the subject
2. 😡 Not sure, Not enough information
3. 😡 Not sure, Not enough information
4. 😡 Yes
5. 😡 Yes
6. 😡 Yes
7. 😡 No, not enough information
8. 😡 Not sure, Don’t know the subject
9. 😡 No, Not enough information
10. 😡 No, Don’t know the subject
11. 😡 No, Don’t know the subject
12. 😡 No, Don’t know the subject
13. 😡 No, Don’t know the subject
14. 😡 No, Low quality, Don’t know the subject
15. 😡 No, Don’t know the subject
16. 😡 No, Don’t know the subject
17. 😡 No, Don’t know the subject
18. 😡 Not sure, Not enough information
19. 😡 No, Don’t know the subject
20. 😡 No, Don’t know the subject
**Task #19 🥰**
- Taps the image for more information
- Mentions that he would use Google search to find more about the image.
**Task #20 🤔**
**Task #21 🥰**
Zooms the image on the detail page.
**Task #22 🥰**
- Was looking for back functionality first
- Swipes back and forth to navigate between suggestions.
**Task #23 🥰**
- Understands the options presented
- Understood that he can select multiple
**Task #24 (instructions)**
**Task #25 🥰**
Understand the options
**Task #26 🥰**
Realizes that he could scroll the article here
**Task #27 😡**
Didn’t see the 'info i' at the top right.
**Task #28 🥰**
More or less understands the concept
**Task #29 (instructions)**
**Task #30 🥰**
Notices the added counter
**Task #31 (instructions)**
**Task #32-36 (see rating scale)**
**Task #37 (open format)**
—
**Final thoughts**
—
##### Follow-up questions
1. it's a random thing and I would say that the overall quality is good.it's a random thing and I would say that the overall quality is good.
2. it's pretty tedious, and it's pretty cumbersome. I guess it has to be because Wikipedia is something that is very serious, and I think it should be taken seriously.
3. pretty hard but time consuming and I would have to say it was well worth it
4. probably not is that I have a weak bladder and I have to use the men's room much much much too often to be a good at any of this. I am sorry take care
---
#### Participant #4: [TestMaster888](https://app.usertesting.com/v/08b9637c-08c6-48e1-a136-51e87355bba9?encrypted_video_handle=aca2c43f-081c-411e-bd46-77be1ae8b991#!/answers)
- Did not notice the 'Suggestion reason'
##### Tasks
**Task #1-5 (instructions)**
**Task #6 🤔**
Doesn’t see the blue tooltip.
**Task #7-8 (instructions)**
**Task #9 🥰**
Describes the tasks properly.
**Task #10 🥰**
Describes the task properly.
**Task #11 🥰**
Associates the robot with a computer program / algorithm.
**Task #12 (instructions)**
**Task #13 🤔**
- Seems like he understands the first onboarding dialog
- Doesn’t react to the second onboarding dialog
**Task #14 (instructions)**
**Task #15 🥰**
- Describes what he sees (article and image)
- Hasn’t really paid attention to the tooltips
**Task #16 🤔**
Hasn’t really read the tooltips.
**Task #17 🥰**
Knows what he needs to do now.
**Task #18 🥰**
- Makes educated choices
- Reads meta information of the image
- Zooms the image intuitively
- Navigates back and forth on detail page and article
- Makes educated decisions
1. 🥰 No, Not relevant
2. 🥰 Yes
3. 🥰 Not sure, Not enough information
4. 🥰 No, Not relevant
5. 🤔 Yes
6. 🤔 Yes
7. 🤔 Yes
8. 🥰 Yes
9. 🤔 Yes
10. 🥰 No, Not relevant
11. 🥰 Yes
12. 🥰 Not sure, Not enough information
13. 🥰 Yes
14. 🥰 No, Not relevant, Low quality
15. 🥰 Not sure, Not enough information
16. 🥰 No, Low quality
17. 🤔 Yes
18. 🥰 Yes
19. 🥰 Not sure, Not enough information
20. 🥰 Yes
**Task #19 🥰**
- Uses file detail page intuitively
- Mentions that he’d Google the image
**Task #20 🤔**
**Task #21 🥰**
Used pinch to zoom intuitively
**Task #22 🥰**
- Taps the button at the top right first
- Then swipes between suggestions
**Task #23 🥰**
Explains the options well
**Task #24 (instructions)**
**Task #25 🥰**
Explains the options well
**Task #26 🥰**
Scrolls down with no issues
**Task #27 🥰**
Taps on the 'info i' icon at the top right
**Task #28 🥰**
Identifies it as an element of positive reinforcement
**Task #29 (instructions)**
**Task #30 🥰**
Sees that the counter has been added
**Task #31 (instructions)**
**Task #32-36 (see rating scale)**
**Task #37 (open format)**
—
**Final thoughts**
—
##### Follow-up questions
1. The algorithm is programmed to look at keywords in the article and do an image search based on the keywords and try to return an image that best matches its search parameters. 65% the images I was presented were relevant.
2. Nothing was confusing. though it might work better if the algorithm presented the human editor with more than 1 image choice.
3. very easy.
4. Yes it would be much better than scouring the web for images manually.
---
#### Participant #5: [Cherry928](https://app.usertesting.com/v/5d186a06-a292-44da-b9e1-3a5719a24a7f?encrypted_video_handle=7b17d7ef-54c3-4f8d-8b32-eb3c6b2351b8#!/answers)
##### Tasks
**Task #1-5 (instructions)**
**Task #6 😡**
- Describes the Explore Feed card
- Reads out the tooltip loud but does not understand it
**Task #7-8 (instructions)**
**Task #9 🥰**
Understands the tasks
**Task #10 🥰**
Understands the task
**Task #11 🥰**
Associates the robot icon with an algorithm.
**Task #12 (instructions)**
**Task #13 🥰**
Understands both onboarding screens
**Task #14 (instructions)**
**Task #15 🥰**
Sees onboarding tooltip and knows what to do.
**Task #16 🥰**
Reads tooltips and understands what to do.
**Task #17 🥰**
Knows what do now.
**Task #18 🥰**
- Does not interact with file detail page at all, probably due to time pressure (mentions that these tests should only be 15mins long)
- Analyzes the information she sees thoroughly
- Scrolls the article intuitively
- “Would you add this image to the article?” is cut off → Display issues on smaller screens
- Selects multiple answers right away
1. 🥰 No, Not relevant
2. 🥰 Yes
3. 🥰 Not sure, Not enough information
4. 🤔 Yes
5. 🥰 Not sure, Not enough information
6. 🥰 Not sure, Not enough information
7. 🥰 Yes
8. 🤔 Not sure, Other
9. 🤔 Not sure, Not enough information
10. 🥰 Not sure, Not enough information
11. 🤔 Yes
12. 🥰 Not sure, Other
13. 🥰 Yes
14. 🥰 No, Low quality
15. 🥰 No, Not relevant
16. 🥰 Not sure, Not enough information
17. 🤔 Yes
18. 🥰 Yes
19. 🥰 Not sure, Not enough information, Don’t know this subject
**Task #19 🥰**
Taps on the image card
**Task #20 🥰**
- Date taken
- Read full image description
- Author (to contact them)
**Task #21 🥰**
- Tries zooming the image from the feed
- Taps on the image and zooms in on detail page
**Task #22 🥰**
Uses swipes gestures
**Task #23 🥰**
Understands the options
**Task #24 (instructions)**
**Task #25 🥰**
Understands the options
**Task #26 🥰**
- Tries to tap the article title
- Also tries to scroll the article
- Taps the info bubble as the last step
**Task #27 🤔**
Would tap the back button and look for a help page (//RS: A possible way to success//)
**Task #28 🥰**
Likely understands it
**Task #29 (instructions)**
**Task #30 🥰**
Sees it and likely understands it
**Task #31 (instructions)**
**Task #32-36 (see rating scale)**
**Task #37 (open format)**
- Easy access to article
- More image information
**Final thoughts**
—
##### Follow-up questions
1. By using key words found in the article title/summary or text. Most suggestions were definitely or likely related, so very good.
2. Easy access to full articles. Description of image shown in full/more of.
3. Fairly easy.
4. Possibly if I had a lot of time on my hands, but probably not in my current situation as I have other things to spend my time on. But if someone is interested in contributing in a fun/easy way, I think this is a good way to do so.