Background

As we prepare for the Alt Text experiment, we want to observe how users interact with the prototypes and surface feedback and opportunities.

Testing groups:

• Experiment Group B: Add an image flow, and then dedicated Alt Text flow after
• Experiment Group C: Text edit in section with image, and then encouragement to add alt-text to that image after publishing

Research Questions

• (B only) After image-related edit is submitted, do they understand they are adding alt-text to the image they just added?
• (C only) After non-image related edit is submitted, do they understand they are making a new edit to the same article?
• Are the tooltips sufficiently helpful?
• Do they understand the alt text guidance?
• Do they know where to go to get additional guidance?
• Are the users able to get sufficient information from the image to write the alt text?
• Are the users able to adequately get contextual information from the article to add alt-text to the image ?
• Do they know how to zoom on the image?
• Do they understand when they are previewing vs when it has been published?
• Is there anything they'd change about the feature?

Target Audience

• Folks in the Americas & Caribbean
• Test in English
• Test in Chinese
  • Not planning to do at this time due to issues with prototype & limited team capacity
• There should be diversity in Gender and Age

Link to protocol

https://docs.google.com/document/d/1pBAbbtJcFlu5ApDIrxknLWPJNysLg4Qoc2Tc7UajP-s/edit

Results

1. Analysis of tasks for English and Chinese tests

Testers for English - Flow B

• Countries: UK (1), Germany (2), India (2)
• Male (4), Female (1)
• 4/5 had used the iOS app before
• 4/5 had contributed to Wikipedia before
• 1/5 had completed a structured edit task before
• 2/5 had previously added alt text to images

Testers for English - Flow C

• Countries: UK (1), Australia (1), India (1), Israel (1), Canada (1)
• Male (2), Female (3)
• 4/5 had used the iOS app before
• 5/5 had contributed to Wikipedia before
• 4/5 had completed a structured edit task before
• 3/5 had previously added alt text to images

Task B.4 Exploring the image recommendations "Add image details" page

English:

• 4/5 understood that they had options to add caption and alt text to the image they just selected "Yes" to
• 1/5 did not scroll, and did not notice the Alt text entry screen
• 1/5 appreciated how we explained the difference between alt text and caption

Task B.7 and C.5 What do you think the modal is asking you? (Add alt text modal)

English flow B:

• 5/5 understood that they were being asked to add alt text to the previous image because it's missing
• 1/5 noted that it would be nice, but not necessary to add
• 1/5 suggested that instead of saying "previous image", it should read "the image you just added"

English flow C:

• 5/5 understood that they were adding alt text for an image from the article you just corrected the misspelled word in.

Task B.10 and C.8 Onboarding: What do you notice? How do you feel about the onboarding information?

English:

• 10/10 thought the onboarding was clear
• 4/10 did not know to scroll through the onboarding & completely missed the last point "Take care with people"
• 3/10 did not understand the "Type of image" point / were confused by the "Type of image" recommendations, and 1 wanted an example
• 1/10 was surprised by the "Type of image" point and found it helpful
• 1/10 wanted an image with an example alt text

Task B.13 and C.11 How do you feel about these tooltips?

English:

• 9/10 thought the tooltips were clear and helpful
• 1/10 thought there should be more guidance about the descriptions, as it will be available to the public
• 1/10 thought they were not enough for how hard this was going to be
• 1/10 recommended that the "review" tooltip could be more detailed and tell or show how you can expand the article (they had missed the "View article" link

Task C.12 Exploring the "Add alt text" page

English:

• 2/5 leave the last tooltip up while exploring
• 2/5 noticed the 125 character limit and thought it was helpful
• 2/5 noticed the guidance link and appreciated it

Task B.15 and C13 Did you find the article summary to be helpful?

English:

• 6/10 agreed that the article summary was helpful
• 1/10 doesn't understand why it will be helpful in writing the alt text
• 1/10 first clicked on the info-i
• 1/10 thought it was the entire article behind, because they didn't see the "view article" link
• 1/10 wants more examples of cacti

Task B.17 and C.14 Please zoom in on the image.

English:

• 5/10 tapped on the image first, then pinch and zoom
• 5/10 would pinch and zoom first on thumbnail, then tap to open
• 1/10 minimized the half sheet and did not know to slide it back up to view the image again

Task B.19 and C.17 Finding additional guidance on how to write alt text

English:

• 6/10 would open the "Guidance for writing alt text" link
• 2/10 would open the info-i next to the image
• 1/10 goes to the overflow menu and tries Learn more and Tutorial, and then finds the guidance link after
• 1/10 minimized the half sheet and did not know to slide it back up to view the image again

Task B.22 and C.29Understanding additional guidance on alt text

English:

• 6/10 thought the guidance did not need any improvements
• 8/10 thought the guidance was short and clear
• 4/10 was happy there were examples linked
• 1/10 was disappointed that it was shorter from the original onboarding guidance, and wishes it was the same. Especially the part about being objective and not make assumptions.
• 1/10 wanted more information on what "Take care when identifying people" meant
• 1/10 wasn't sure if they should include scientific jargon or latin names in Alt text and wanted guidance on that.
• 1/10 wanted more guidance on length
• 1/10 wanted more information about when to include words and other information from the image

Task B.24 and C.22Finding additional information about the image

English:

• 6/10 knew to click on the info-i next to the image, 1/10 found it after being prompted.
• 1/10 tapped on the image first, then found the info-i button
• 1/10 read through article summary and clicked "View article" to find more information
• 1/10 had minimized the half sheet and did not know to slide it back up to view the image again. She would search for image on Google Lens
• 1/10 suggested that the info-i could be placed closer to the image, they weren't sure if it was related to the image or alt text guidance
• 1/10 suggested that the tool tip actually point to the info-i button (tooltip points to image thumbnail)

Task B.27 and C.25Reading additional information about the image

English:

• 6/10 felt the information was complete and not missing anything important
• 2/10 wished that it included the person responsible for creating the image, who drew the piece
• 2/10 wanted more details about"Date", whether it was date created, uploaded, and what the month and day was in addition to year
• 1/10 wanted to know where the information came from, if it was AI-generated or edited by humans (they assumed AI generated because there wasn't an opportunity to add "depicted persons" for example in the Add an Image task).
• 1/10 had minimized the half sheet and did not know to slide it back up to view the image again.
• 1/10 noticed that there is more information here than in the full-screen view of the image
• 1/10 wanted to know more about the camera metadata (mobile or digital camera)

3. Analysis follow-up questions

Question B.34 and C.32How was the process of adding alt text to an image?

English: flow B:

• 2/5 answered Easy
• 3/5 answered Medium, one because she couldn't figure out how to scroll up the halfsheet

English: flow C:

• 5/5 answered Easy

Question B.35How did you respond to the modal asking if this should be a separate task?

English:

• 3/5 answered Yes, that adding alt text should be a separate task. However 2 seemed to agree that the task should remain as they just completed it, pointing to a lack of clarify in the question. 1 said it should be separate so people don't get confused.
• 1/5 answered No, it should be necessary, not optional (they interpreted "suggested" as optional.
• 1/5 did not answer

Question C.33How did you respond to the modal asking about your satisfaction with this feature?

English:

• 4/5 answered Satisfied
• 1/5 answered Neutral, didn't think it was anything special
• 3/5 appreciated all of the guidance, steps, and pointers.
• 1/5 thought it could use a few more examples and points of guidance

Question B.36 and C.34How would you explain the concept of alt text to a friend?

English:

• 9/10 were able to summarize the task
• 4/10 mentioned that it improves articles for those who are visually impaired
• 1/10 described it as a guided tour to help users who are not experienced yet

Question B.37 and C.35How was the guidance presented in the workflow?

English:

• 9/10 answered Very Helpful
• 1/10 answered Helpful
• 1/10 reiterated that they with the guidance at the beginning of the workflow was available later on / standardized with guidance later on
• 1/10 who has experience writing alt text thought this was very clear for folks who haven't done it before

Question B.38 and C.36Did you find the information about the image that you expected?

English:

• 10/10 answered Yes they found the information about the image they expected
• 1/10 said they found more than expected
• 1/10 reiterated that because we weren't able to edit it the image information when we added the image, they want to know where it comes from

Question B.39 and C.37Do you think you would be able to add alt text to a new image?

English:

• 10/10 answered Yes
• 1/10 noted that the step by step guide was very useful"

Question B.40 and C.38 Suggestions for improvements

English:

• 1/10 asked that we make it clearer what is AI generated vs not. Source of information about the image.
• 1/10 suggested we standardize the advice given
• 1/10 would like explanation phrases to be clearer
• 1/10 would like examples in the first onboarding to clearly set expectations
• 1/10 would like the image info icon to be bigger
• 1/10 would like to see location in image details
• 1/10 would like to see colors in image details

4. Summary:

• There was clear understanding on what the alt text prompt was asking them to do
  • For flow B: 5/5 understood that they were being asked to add alt text to the previous image because it's missing
  • For flow C: 5/5 understood that they were adding alt text for an image from the article you just corrected the misspelled word in.
• The onboarding, tooltips, and guidance were clear and helpful to users, with some room for small improvements
  • 10/10 thought the onboarding was clear
  • 9/10 thought the tooltips were clear and helpful
• Users would be able to zoom, but their first choice was not supported
  • 5/10 tapped on the image first, then pinch and zoom
  • 5/10 would pinch and zoom first on thumbnail, then tap to open
• Most users could find the guidance they needed
  • 6/10 would open the "Guidance for writing alt text" link
• Users could find additional information about the image
  • 6/10 knew to click on the info-i next to the image, 1/10 found it after being prompted.
  • 1/10 tapped on the image first, then found the info-i button -Most testers thought the image information provided was complete, while others noted room for small improvements
  • 6/10 felt the information was complete and not missing anything important
• Testers understood the task, and were able to successfully summarize it afterwards
• All testers felt confident they could add alt text to a new image

5. Recommendations

(Updated so far with recommendations from English)

Required

• Standardize the in-flow guidance to match onboarding text. The guidance should include the 5 points from the onboarding, and include a link to examples (design)
• Add arrow to make it clear that folks need to scroll through onboarding, or reformat into 2 slides of onboarding (design)
• Allow user to zoom on image thumbnail without clicking (engineering)
• Add Author in Image Information if it's available for that image (G. Mütrel, Leipzig ; Berlin ; Wien : F.A. Brockhaus in example image) (design)
• Move commons link up to top of Image Information, and change wording to match Wikimedia commons language: "This is a file from the Wikimedia Commons. Information from its description page there is shown below. (design)
• Update text of survey for B: Would you rather use a feature that provided a feed of images in need of alt text? (Y/N, if no allow for text reason) (product)
• Add second survey for C that will come after satisfaction survey, with text: We need your feedback. Would you like to be notified about suggested edits on articles after you edit them? (Y/N, if no allow for text reason) (design)

Nice-to-have

• Within the "Add alt text" modal, change "previous image" to "the image you just added" (design)
• Point the image information tooltip to the info-i and not the image (design)
• Within Image Information, change "Date" to "Date created or first published", to clarify for users and match Commons (design)
• Provide an example of how to not start with "type of image" in "Type of image" onboarding bullet point (product, design - update later when accessibility org reviews text )

---

References: Prototype Protocol