### Background
As we prepare for the Alt Text experiment, we want to observe how users interact with the prototypes and surface feedback and opportunities.
**Testing groups:**
- Experiment Group B: Add an image flow, and then dedicated Alt Text flow after
- Experiment Group C: Text edit in section with image, and then encouragement to add alt-text to that image after publishing
####Research Questions
- (B only) After image-related edit is submitted, do they understand they are adding alt-text to the image they just added?
- (C only) After non-image related edit is submitted, do they understand they are making a new edit to the same article?
- Are the tooltips sufficiently helpful?
- Do they understand the alt text guidance?
- Do they know where to go to get additional guidance?
- Are the users able to get sufficient information from the image to write the alt text?
- Are the users able to adequately get contextual information from the article to add alt-text to the image ?
- Do they know how to zoom on the image?
- Do they understand when they are previewing vs when it has been published?
- Is there anything they'd change about the feature?
#### Target Audience
- [ ] Folks in the Americas & Caribbean
- [ ] Test in English and Chinese
- [ ] There should be diversity in Gender and Age
#### Link to protocol
https://docs.google.com/document/d/1pBAbbtJcFlu5ApDIrxknLWPJNysLg4Qoc2Tc7UajP-s/edit
## Results
### 1. Analysis of tasks for English and Chinese tests
**Testers for English - Flow B**
- Countries: UK (1), Germany (2), India (2)
- Male (4), Female (1)
- 4/5 had used the iOS app before
- 4/5 had contributed to Wikipedia before
- 1/5 had completed a structured edit task before
- 2/5 had previously added alt text to images
**Testers for English - Flow C**
- Countries: UK (1), Australia (1), India (1), Israel (1), Canada (1)
- Male (2), Female (3)
- 4/5 had used the iOS app before
- 5/5 had contributed to Wikipedia before
- 4/5 had completed a structured edit task before
- 3/5 had previously added alt text to images
##### `Task B.4` Exploring the image recommendations "Add image details" page
English:
- 4/5 understood that they had options to add caption and alt text to the image they just selected "Yes" to
- 1/5 did not scroll, and did not notice the Alt text entry screen
- 1/5 appreciated how we explained the difference between alt text and caption
##### `Task B.7 and C.5` What do you think the modal is asking you? (Add alt text modal)
English flow B:
- 5/5 understood that they were being asked to add alt text to the previous image because it's missing
- 1/5 noted that it would be nice, but not necessary to add
- 1/5 suggested that instead of saying "previous image", it should read "the image you just added"
English flow C:
- 5/5 understood that they were adding alt text for an image from the article you just corrected the misspelled word in.
##### `Task B.10 and C.8` Onboarding: What do you notice? How do you feel about the onboarding information?
English:
- 10/10 thought the onboarding was clear
- 4/10 did not know to scroll through the onboarding & completely missed the last point "Take care with people"
- 3/10 did not understand the "Type of image" point / were confused by the "Type of image" recommendations, and 1 wanted an example
- 1/10 was surprised by the "Type of image" point and found it helpful
- 1/10 wanted an image with an example alt text
##### `Task B.13 and C.11` How do you feel about these tooltips?
English:
- 9/10 thought the tooltips were clear and helpful
- 1/10 thought there should be more guidance about the descriptions, as it will be available to the public
- 1/10 thought they were not enough for how hard this was going to be
- 1/10 recommended that the "review" tooltip could be more detailed and tell or show how you can expand the article (they had missed the "View article" link
##### `Task C.12` Exploring the "Add alt text" page
English:
- 2/5 leave the last tooltip up while exploring
- 2/5 noticed the 125 character limit and thought it was helpful
- 2/5 noticed the guidance link and appreciated it
##### `Task B.15 and C13` Did you find the article summary to be helpful?
English:
- 6/10 agreed that the article summary was helpful
- 1/10 doesn't understand why it will be helpful in writing the alt text
- 1/10 first clicked on the info-i
- 1/10 thought it was the entire article behind, because they didn't see the "view article" link
- 1/10 wants more examples of cacti
##### `Task B.17 and C.14` Please zoom in on the image.
English:
- 5/10 tapped on the image first, then pinch and zoom
- 5/10 would pinch and zoom first on thumbnail, then tap to open
- 1/10 minimized the half sheet and did not know to slide it back up to view the image again
##### `Task B.19 and C.17` Finding additional guidance on how to write alt text
English:
- 6/10 would open the "Guidance for writing alt text" link
- 2/10 would open the info-i next to the image
- 1/10 goes to the overflow menu and tries Learn more and Tutorial, and then finds the guidance link after
- 1/10 minimized the half sheet and did not know to slide it back up to view the image again
##### `Task B.22 and C.29`Understanding additional guidance on alt text
English:
- 6/10 thought the guidance did not need any improvements
- 8/10 thought the guidance was short and clear
- 4/10 was happy there were examples linked
- 1/10 was disappointed that it was shorter from the original onboarding guidance, and wishes it was the same. Especially the part about being objective and not make assumptions.
- 1/10 wanted more information on what "Take care when identifying people" meant
- 1/10 wasn't sure if they should include scientific jargon or latin names in Alt text and wanted guidance on that.
- 1/10 wanted more guidance on length
- 1/10 wanted more information about when to include words and other information from the image
##### `Task B.24 and C.22`Finding additional information about the image
English:
- 6/10 knew to click on the info-i next to the image, 1/10 found it after being prompted.
- 1/10 tapped on the image first, then found the info-i button
- 1/10 read through article summary and clicked "View article" to find more information
- 1/10 had minimized the half sheet and did not know to slide it back up to view the image again. She would search for image on Google Lens
- 1/10 suggested that the info-i could be placed closer to the image, they weren't sure if it was related to the image or alt text guidance
- 1/10 suggested that the tool tip actually point to the info-i button (tooltip points to image thumbnail)
##### `Task B.27 and C.25`Reading additional information about the image
English:
- 6/10 felt the information was complete and not missing anything important
- 2/10 wished that it included the person responsible for creating the image, who drew the piece
- 2/10 wanted more details about"Date", whether it was date created, uploaded, and what the month and day was in addition to year
- 1/10 wanted to know where the information came from, if it was AI-generated or edited by humans (they assumed AI generated because there wasn't an opportunity to add "depicted persons" for example in the Add an Image task).
- 1/10 had minimized the half sheet and did not know to slide it back up to view the image again.
- 1/10 noticed that there is more information here than in the full-screen view of the image
- 1/10 wanted to know more about the camera metadata (mobile or digital camera)
#### 3. Analysis follow-up questions
##### `Question B.34 and C.32`How was the process of adding alt text to an image?
English: flow B:
- 2/5 answered Easy
- 3/5 answered Medium, one because she couldn't figure out how to scroll up the halfsheet
English: flow C:
- 5/5 answered Easy
##### `Question B.35`How did you respond to the modal asking if this should be a separate task?
English:
- 3/5 answered Yes, that adding alt text should be a separate task. However 2 seemed to agree that the task should remain as they just completed it, pointing to a lack of clarify in the question. 1 said it should be separate so people don't get confused.
- 1/5 answered No, it should be necessary, not optional (they interpreted "suggested" as optional.
- 1/5 did not answer
##### `Question C.33`How did you respond to the modal asking about your satisfaction with this feature?
English:
- 4/5 answered Satisfied
- 1/5 answered Neutral, didn't think it was anything special
- 3/5 appreciated all of the guidance, steps, and pointers.
- 1/5 thought it could use a few more examples and points of guidance
##### `Question B.36 and C.34`How would you explain the concept of alt text to a friend?
English:
- 9/10 were able to summarize the task
- 4/10 mentioned that it improves articles for those who are visually impaired
- 1/10 described it as a guided tour to help users who are not experienced yet
##### `Question B.37 and C.35`How was the guidance presented in the workflow?
English:
- 9/10 answered Very Helpful
- 1/10 answered Helpful
- 1/10 reiterated that they with the guidance at the beginning of the workflow was available later on / standardized with guidance later on
- 1/10 who has experience writing alt text thought this was very clear for folks who haven't done it before
##### `Question B.38 and C.36`Did you find the information about the image that you expected?
English:
- 10/10 answered Yes they found the information about the image they expected
- 1/10 said they found more than expected
- 1/10 reiterated that because we weren't able to edit it the image information when we added the image, they want to know where it comes from
##### `Question B.39 and C.37`Do you think you would be able to add alt text to a new image?
English:
- 10/10 answered Yes
- 1/10 noted that the step by step guide was very useful"
##### `Question B.40 and C.38` Suggestions for improvements
English:
- 1/10 asked that we make it clearer what is AI generated vs not. Source of information about the image.
- 1/10 suggested we standardize the advice given
- 1/10 would like explanation phrases to be clearer
- 1/10 would like examples in the first onboarding to clearly set expectations
- 1/10 would like the image info icon to be bigger
- 1/10 would like to see location in image details
- 1/10 would like to see colors in image details
#### 4. Recommendations
(Updated so far with recommendations from English)
**Required**
- Standardize the in-flow guidance to match onboarding text. The guidance should include the 5 points from the onboarding, and include a link to examples (design)
- Add arrow to make it clear that folks need to scroll through onboarding, or reformat into 2 slides of onboarding (design)
- Allow user to zoom on image thumbnail without clicking (engineering)
- Add Author in Image Information if it's available for that image ([[ https://en.wikipedia.org/wiki/File:Various_Cactaceae.jpg | G. Mütrel, Leipzig ; Berlin ; Wien : F.A. Brockhaus ]] in example image) (design)
- Move commons link up to top of Image Information, and change wording to match Wikimedia commons language: "This is a file from the Wikimedia Commons. Information from its [[ https://en.wikipedia.org/wiki/File:Various_Cactaceae.jpg | description page ]] there is shown below. (design)
- Update text of survey for B: Would you use a feature that provided a feed of images in need of alt text? (Y/N, if no allow for text reason) (product)
- Add second survey for B, with text: We need your feedback. Would you like to be notified about suggested edits on articles after you edit them? (Y/N, if no allow for text reason) (design)
**Nice-to-have**
- Within the "Add alt text" modal, change "previous image" to "the image you just added" (design)
- Point the image information tooltip to the info-i and not the image (design)
- Within Image Information, change "Date" to "Date created or first published", to clarify for users and match Commons (design)
- Provide an example of how to not start with "type of image" in "Type of image" onboarding bullet point (product, design - update later when accessibility org reviews text )
-------------
References: [[ https://docs.google.com/document/d/18HQW_dc-TcBGWu-Y8WRkZTqcsckbfWJQxCru1EiURpI/edit | Prototype Protocol ]]