Page MenuHomePhabricator

[SPIKE] Investigate image captions alt text impact on image captions suggested edits
Closed, ResolvedPublic

Description

Background
@RHo shared in Slack: Great to see in the state of communities newsletter that there is now a specific wikidata ALT text property for images on Commons, finally closing out T166094.
Posting here as I am considering how this may change Growth’s Add image task, and wondering if this could have bearing on changes to Android’s Suggested edits image caption task too.

Task

  • Investigate if alt text will override changes made through Android app suggested edits
  • Consider how these changes will be applied for English, Swahili, Arabic, if there is a difference
  • Provide recommendations based on research

Event Timeline

Some of the conversations I am following as part of this spike:

Basics on how alt texts are being planned to be added, stored and retrieved: https://phabricator.wikimedia.org/T325955
Possibility of making alt text available through imageInfo api, which we have already been using for license and other image related metadata: https://phabricator.wikimedia.org/T325948
Currently monolingual and working towards making it a multilingual field [trying to understand this still]: https://phabricator.wikimedia.org/T325944
Decision to keep alt text separate from captions: https://phabricator.wikimedia.org/T322319

Miscellaneous :
https://phabricator.wikimedia.org/T325015

Will write up a report after following up on a couple things tomorrow.

Basic understanding of alt_text: It is different from captions as in caption might read "Rose", while the alt_text will read "A red flower with thorns on the stem on a green background". Refer: "the American Foundation of the Blind flagged missing alt texts as an accessibility issue" on T323634

Investigate if alt text will override changes made through Android app suggested edits

alt_text has been created as a separate property on wikidata, from captions which is labels property on wikidata. These properties are then used on structured data in commons.
Since we are only consuming or updating captions [aka labels], this will in no way override the alt_text. So in short our SE Image captions feature poses no threat.

Consider how these changes will be applied for English, Swahili, Arabic, if there is a difference
It has been decided to have this property support many languages. Discussions about the data type and how it will be served are still on-going on this task : T325944
However, there is a clear understanding that once the alt_text property is created and used on commons, it will be possible to enter values for it in different languages, and the logical expectation is that it will be served to us through the imageInfo api with multilingual support just like how the captions are served currently. That is, there will be a default lang [en], and then all the languages that the alt_text is available in, and we can use them as we need.

Provide recommendations based on research

alt_text is a very welcome improvement for screen readers, so a huge win for accessibility. It is not the same as captions [See this discussion to understand why: T322319#8372100
After the api is available for alt_text, recommendations on how it is useful to us:

  1. When images are used inside an article, since it is served within the html, it will be automatically available in the alt field and will be picked up by screen readers.
  2. When we are showing a commons image natively - Gallery or file page - we are already making calls to the image_info api to get some metadata on the image to display, and when alt_text is available, we can update the content_description of the image with its alt_text and Android screen readers will pick that up.

@Seddon Thank you for pointing me to the right tasks
@Dbrant As always, thank you for demystifying the wikidata structured data tangles :-)