| Sebastian_Berlin-WMSE | |
| May 31 2017, 1:18 PM |
| F72829477: image.png | |
| Mar 13 2026, 8:22 AM |
| F72807246: image.png | |
| Mar 11 2026, 3:15 PM |
Figure out how to recite images. The alt attribute can be used for this, but may not always be present. Also, how should caption be handled in relation to alt text.
The placement of images is not necessarily related to the text or their position in the wikitext/HTML.
| Subject | Author | Repo | Branch | Lines +/- | |
|---|---|---|---|---|---|
| Read alt-text on images | Viktoria Hillerud WMSE | mediawiki/extensions/Wikispeech | master | +164 -4 | |
| Read images | Viktoria Hillerud WMSE | mediawiki/extensions/Wikispeech | master | +1 -2 |
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Open | None | T388096 ☂ Read parts of content | |||
| Resolved | Viktoria_Hillerud_WMSE | T166670 Recite images |
Change #1248018 had a related patch set uploaded (by Viktoria Hillerud WMSE; author: Viktoria Hillerud WMSE):
[mediawiki/extensions/Wikispeech@master] Read images
A first implementation, and patch, is to just remove the "figure": true from "WikispeechRemoveTags" in the config, then it reads the alt-text IF there is any alt-text (otherwise it just stops)
Therefore, some things to think about further is:
I realized that images can also occur in galleries of different sorts, and therefore everyting with the class gallerytext should also be injected with text if missing caption.
I will look into how to best do this, but I am thinking that a parameter should be sent t the wikispeech-segment API, to check if there is missing captions. Will investigate this further.
I will look into how to best do this, but I am thinking that a parameter should be sent t the wikispeech-segment API, to check if there is missing captions. Will investigate this further.
When thinking more about this, it could be enough just to handle the empty captions entirely in the backend. There could be a check wheter part-of-content is true, then fill the empty captions with a standard caption of some sort.
I also discovered that it already reads galleries captions, because it was only <figure> tag that was removed in removeTags
Since Wikispeech builds segments from the DOM and empty captions (figcaption and .gallerytext with no text) contain no text nodes, they never become CleanedText and therefore never appear in the segment list.
That means that there is nothing to read out loud. To fix this, I inject a fallback text node in the DOM in Cleaner::addContent().
When the element is a figcaption or has class .gallerytext, I append a fallback text node. That way, the caption actually gets a text node, so the segmentation and TTS will read it too.
I thought this was if there were no captions at all, or alt text for that matter, the user can still get information about that it is missing, but maybe this was not what we wanted?
I thought that would be covered by announcing the image element (when announcing POC is on). Can you give an example of what you're describing?
With POC on, Wikispeech will announce the image/link itself, but it does not announce the caption slot when the caption is empty.
For example: in a gallery:
We decided that only focus on reading the alt-text if it's present. It already reads captions if present.
One thing to think about is that if both alt-text and caption are present, it will read both, even if they're exactly the same. That's not an issue if they're not the same but in galleries, users can choose to have the same alt-text as the caption, and then it will read the same text twice:
An option here could be to exclude one of them to be read, if the text are the same. Easiest would probably be then to not read the alt-text. What do you think? @Sebastian_Berlin-WMSE
I think it makes sense to only read one of alt text and caption if they are identical. At least if it's easy to do. If it proves tricky for some reason I'd say skip it for now and see what feedback we get.
Change #1251300 had a related patch set uploaded (by Viktoria Hillerud WMSE; author: Viktoria Hillerud WMSE):
[mediawiki/extensions/Wikispeech@master] Read image captions
Change #1248018 merged by Viktoria Hillerud WMSE:
[mediawiki/extensions/Wikispeech@master] Read images
I tried it now and it mostly works fine. Just two things:
[[File:Portrait of a father.jpg|thumb|alt=A small ape on a big ape.|Two apes.]]
will result in the utterance:
" link: start of image: alt text: A small ape on a big ape.: Two apes."
There is a colon, but it not that noticeable.
- When there's both alt text and caption the latter is not announced. This may make it unclear when it starts and the alt text ends. For instance this wikitext:
[[File:Portrait of a father.jpg|thumb|alt=A small ape on a big ape.|Two apes.]]will result in the utterance:
" link: start of image: alt text: A small ape on a big ape.: Two apes."There is a colon, but it not that noticeable.
Aha, so maybe it should be more like: " link: start of image: alt text: A small ape on a big ape.: caption: Two apes."?
- Alt texts aren't read in a gallery. I don't think galleries need to be supported at this stage and that you can save them for a separate patch if you want.
Ah, yes that's true. Maybe I'll do them in a separate task even, if I'll end up not adding them in this patch.
Yes, that sounds good. You could also maybe use "alternativ text" rather than "alt text". Even if they mean the same thing, the latter is more of a technical term. I also saw the "Alternative text" is what it's called in Visual editor's UI.
Change #1251300 merged by Viktoria Hillerud WMSE:
[mediawiki/extensions/Wikispeech@master] Read alt-text on images