Test wiki created on Patch demo by LWatson-WMF using patch(es) linked to this task:
https://b90e6d072a.catalyst.wmcloud.org/wiki/Main Page

Test wiki created on Patch demo by LWatson-WMF using patch(es) linked to this task:
https://7babc92315.catalyst.wmcloud.org/wiki/Main Page

Test wiki on Patch demo by LWatson-WMF using patch(es) linked to this task was deleted:

https://b90e6d072a.catalyst.wmcloud.org/w/

Test wiki on Patch demo by LWatson-WMF using patch(es) linked to this task was deleted:

https://7babc92315.catalyst.wmcloud.org/w/

These are the caption text fallback options in this order:

1. figcaption
2. alt text
3. file name

Questions that arose in Slack:

• Images that don't have a figcaption or alt text will display the file name. The file name is not a suitable option to include in the fallback. What should happen when images don't have a figcaption or alt text?
• How should the nearby paragraph text fit into the fallback logic?
  • Should the paragraph text be included in the fallback options or replace the logic entirely?
• Do we display paragraph text in DetailView and VTOC?

What about infobox images? How should they be handled?

• for now, adjacent paragraph is okay. We'll review this in a follow-up ticket

I handled infoboxes in an earlier change, but removed it ¯\_(ツ)_/¯. I need to address this in the patch. Infobox image caption text will be the first paragraph in the lead section. Is that okay?

An update to answer some questions mentioned:

• The nearby paragraph text will appear in the DetailView and VTOC components.
• Fallback options: The paragraph text is added as a fallback option (first tries figcaption > paragraph > alt text > file name). It does not replace figcaption, alt text, and file name. The order of options can be updated in the useImageCaption composable.
• Infobox images: Infobox images are typically placed in the lead section of the article content before the leading paragraph. They are the first set of images in the horizontal scroll (Carousel). To avoid displaying an unhelpful image file name as the caption text, the infobox images that don't have a figcaption or alt text will use the first paragraph from the article content.
  • Note: It is out of scope to capture relevant image text from table images, such as infoboxes.

Test wiki created on Patch demo by LWatson-WMF using patch(es) linked to this task:
https://9ffa99b619.catalyst.wmcloud.org/wiki/Main Page

Design requirements for DetailView:

• Max height 80% of viewport to prevent scrolling
• Truncate at sentence boundaries when possible

When handing off to QA, I noticed this CORS issue in the patchdemo link https://9ffa99b619.catalyst.wmcloud.org/wiki/Paris which makes the DetailView caption text unreadable. There is a merged fix in the patchdemo repo, but we are waiting for its deployment. After it's deployed, the background gradient tool will work as expected, and text will be readable.

Test wiki created on Patch demo by EGardner (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/85fc46bc00/w/

Hey @JScherer-WMF – I moved this task to "Needs QA" but I assigned it to you because I'd like some design input here.

Are you okay with the current behavior that uses the first paragraph of a given article as the text when images from an article Infobox lack both caption and alt text? In practice this means that on many articles, the users will be presented with many images that simply restate the first paragraph again and again (and these will be the first things the user sees in the VTOC). I think this might lead folks to conclude that the feature is broken.

You can see an example of the current behavior here: https://patchdemo.wmcloud.org/wikis/85fc46bc00/wiki/Paris – I suspect this will be an issue with many well-illustrated articles.

Test wiki on Patch demo by LWatson-WMF using patch(es) linked to this task was deleted:

https://9ffa99b619.catalyst.wmcloud.org/w/

There's a typo in the URL shared. Try this https://patchdemo.wmcloud.org/wikis/85fc46bc00/wiki/Paris or remove the extra characters at the end

I've gone ahead and +2ed the patch, but I still think we should do some design review next week and decide whether or not we think the repeating first paragraph is going to seem jarring to users. We can file a follow-up task to address that if necessary.

Change #1180687 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: basic approach to nearest paragraph

https://gerrit.wikimedia.org/r/1180687

We perform some review/QA of this using a range of different articles

I wonder if this makes sense to do not just for this ticket, but for the Image Browsing prototype as a whole. If so, then I will move this into sign off and we can create a dedicated task to handle a more extensive QA audit of our feature in a week or two.

Change #1186061 had a related patch set uploaded (by LWatson; author: LWatson):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing:

https://gerrit.wikimedia.org/r/1186061

Change #1186066 had a related patch set uploaded (by Eric Gardner; author: Eric Gardner):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: Ensure the "more" button updates along with caption

https://gerrit.wikimedia.org/r/1186066

Change #1186061 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: update useImageCaption param

https://gerrit.wikimedia.org/r/1186061

Change #1186066 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: Ensure the "more" button updates along with caption

https://gerrit.wikimedia.org/r/1186066

This merged change is related to this ticket https://gerrit.wikimedia.org/r/c/mediawiki/extensions/ReaderExperiments/+/1185985
I made an error in the commit message by adding an extra empty line between the "bug" and "change-id" :-/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/85fc46bc00/w/

In terms of QA process for this task, I would recommend the following:

• Navigate to this patchdemo instance: https://patchdemo.wmcloud.org/wikis/7b60cdc924/wiki/Berlin
• Open up the Image Browsing feature by clicking on a carousel image at the top of an article page
• Scroll below the detail view and look at the visual table of contents section. Look at the text that accompanies each image in the grid here and note any irregularities, mismatches, etc.
• Do this check for a representative sampling of articles

We know that users will tend to see the first paragraph repeated in many situations (this is based on how we are handling images in the infobox which lack both captions as well as alt text) – that's a known limitation.

In T402223#11150068, @egardner wrote:

Hey @JScherer-WMF – I moved this task to "Needs QA" but I assigned it to you because I'd like some design input here.

Are you okay with the current behavior that uses the first paragraph of a given article as the text when images from an article Infobox lack both caption and alt text? In practice this means that on many articles, the users will be presented with many images that simply restate the first paragraph again and again (and these will be the first things the user sees in the VTOC). I think this might lead folks to conclude that the feature is broken.

You can see an example of the current behavior here: https://patchdemo.wmcloud.org/wikis/85fc46bc00/wiki/Paris – I suspect this will be an issue with many well-illustrated articles.

Thanks for flagging this @egardner @lwatson. This does indeed look broken to see the first paragraph over and over in the vtoc. I think for this usecase, seeing nothing in the caption area for the images that would repeat the same text as the previous image would be preferable to seeing the same text repeated.

In T402223#11164857, @JScherer-WMF wrote:

Thanks for flagging this @egardner @lwatson. This does indeed look broken to see the first paragraph over and over in the vtoc. I think for this usecase, seeing nothing in the caption area for the images that would repeat the same text as the previous image would be preferable to seeing the same text repeated.

All other images in the VTOC will have some text associated with them. Will it be strange when the first few images lack that information? We could try to show just the image title as a fallback (with the caveat that this is not always very helpful or informative). I'm not really sure what other data we have that we can rely on.

In T402223#11164908, @egardner wrote:

In T402223#11164857, @JScherer-WMF wrote:

Thanks for flagging this @egardner @lwatson. This does indeed look broken to see the first paragraph over and over in the vtoc. I think for this usecase, seeing nothing in the caption area for the images that would repeat the same text as the previous image would be preferable to seeing the same text repeated.

All other images in the VTOC will have some text associated with them. Will it be strange when the first few images lack that information? We could try to show just the image title as a fallback (with the caveat that this is not always very helpful or informative). I'm not really sure what other data we have that we can rely on.

If title is reliably available, then that works. I agree that something unique is better than nothing, and nothing is better than the repeated paragraph.

@lwatson do you feel like you have time to make this update? If so I'll move this back into the "doing" column, and you can file a new patch against the same task number. Should just be a matter of tweaking the logic in thumbExtractor.js to remove special handling for infobox images.

@egardner Yes, I'll open a new patch to make the update and remove infobox handling.

I'd prefer not to use the file name because that text can be unhelpful and would appear in the first few visible examples. I thought the first paragraph would be a better option, even though it's repeated in some cases.

The Minerva skin sets padding-bottom to paragraph elements, which causes the clipped content to partially show. What's the best approach to override the Minerva styles?

.content p {
    padding-bottom: 0.5em;
    margin: 0.5em 0 0 0;
}

Change #1186610 had a related patch set uploaded (by LWatson; author: LWatson):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: remove first paragraph for infobox images

https://gerrit.wikimedia.org/r/1186610

In T402223#11165141, @lwatson wrote:

The Minerva skin sets padding-bottom to paragraph elements, which causes the clipped content to partially show. What's the best approach to override the Minerva styles?

.content p {
    padding-bottom: 0.5em;
    margin: 0.5em 0 0 0;
}

@lwatson I'd say we should just prevent that style from applying here.

Test wiki created on Patch demo by EGardner (WMF) using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/a07111eaec/w/

Change #1186615 had a related patch set uploaded (by LWatson; author: LWatson):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: override Minerva skin paragraph styles

https://gerrit.wikimedia.org/r/1186615

Change #1186615 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: override Minerva skin paragraph styles

https://gerrit.wikimedia.org/r/1186615

Change #1186642 had a related patch set uploaded (by LWatson; author: LWatson):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: remove file name from fallback options

https://gerrit.wikimedia.org/r/1186642

Test wiki created on Patch demo by LWatson-WMF using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/49eab22e2b/wiki/Main Page

Change #1186646 had a related patch set uploaded (by LWatson; author: LWatson):

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: nearby paragraph as the first fallback option

https://gerrit.wikimedia.org/r/1186646

Test wiki created on Patch demo by LWatson-WMF using patch(es) linked to this task:
https://patchdemo.wmcloud.org/wikis/24832aef52/wiki/Main Page

copied over

In T401221#11167810, @ovasileva wrote:

Another potential issue is that we are showing the paragraph in the detail view as well, which I think needs to be always the caption, independent of our decision on what to do in the visual ToC view. @JScherer-WMF - could you confirm?

Yeah, this is something that Olga and I discussed in a 1:1 a couple weeks ago that didn't get communicated out properly. Apologies for that.
Here's my thinking here:

There are basically three different information hierarchies here:

1. images accompany text (article view)
2. text accompanies images (vtoc)
3. text describes images (captions)

The issue with using the paragraph as the sole text in the detail view is that the paragraph alone often doesn't describe the image adequately. If the image is in the detail view, you want text that describes images (number 3 above), and only the caption does that adequately. What we're trying to test is whether that 2nd text/image relationship is effective for readers' information needs.

So there are two options in cases when we have both a paragraph and a caption:

1. we only show caption in the detail view, only show paragraph in the vtoc.
   • the inconsistency makes me a bit itchy here.
2. We concatenate them in both the detail view and the vtoc when they're both available.
   • this makes them consistent, and adds more information.
   • this would be increased scope, though, I assume.

<caption> | <paragraph>

e.g.

Paragraph is underlined for emphasis to differentiate them.

We probably need different design treatments for captions and paragraph text, which I'll look at for the next revision of the feature.

Ideally the thing we're trying to test in the vtoc is whether or not adjacent paragraph text assists information seeking and retrieval or not. Ultimately, I think we should go with the treatment that gives us the best signal for this. In this case, clicking "more" in the detail view gives us this signal, as does engagement with the vtoc in general. @ovasileva what do you think?

Summary from internal conversation:

• Remove special handling for infobox images that don't have a figcaption or alt text due to concerns about repeating the first paragraph for those infobox images. (Note: infobox images are the first few images in this new image browsing feature.)

• Provide POC options to determine the approach for MVP:
  • Option 1: File name - displays the file name when figcaption or alt text are unavailable (patch 1186610 | patchdemo)
  • Option 2: No text - displays no caption text when figcaption and alt text are unavailable (patch 1186642 | patchdemo)
  • Option 3: Paragraph fallback - caption text tries the nearby paragraph first, then figcaption, alt text, and finally file name. This could be helpful for focusing/testing the quality of the basic nearby paragraph approach. (patch 1186646 | patchdemo)
  • Option 4: Horizontal scroll grouping - groups VTOC images with the same paragraph text in a horizontal scroll (patch | patchdemo)
  • Option 5: Masonry grid grouping - groups VTOC images with the same paragraph text in a masonry grid (patch | patchdemo)

• Decision: Implement the "no text" option for images that don't have figcaption or alt text, and remove the file name from caption fallback options. The file name is more than likely unhelpful, and based on our current image data, showing no text is better than displaying the file name.

• Future versions will explore options that group VTOC images with the same paragraph text. (Thanks to @matthiasmullie for quickly prototyping grouping the VTOC images in options 4 and 5)

Proposal after conversation with @egardner:

For the detail view

1. let's go with the caption only (it's a bit too much scope creep to show both caption and paragraph)
2. If no caption is available, go to alt. If no alt, nothing

For the ToC view:

• Exclude infoboxes from receiving text
• For all other images, use paragraph
• In addition, create QA ticket for @Edtadros and @Etonkovidova to review top 50 articles and note inconsistencies between paragraph text and image

@JScherer-WMF - how does this sound?

In T402223#11169347, @ovasileva wrote:

Proposal after conversation with @egardner:

For the detail view

1. let's go with the caption only (it's a bit too much scope creep to show both caption and paragraph)
2. If no caption is available, go to alt. If no alt, nothing

For the ToC view:

• Exclude infoboxes from receiving text
• For all other images, use paragraph
• In addition, create QA ticket for @Edtadros and @Etonkovidova to review top 50 articles and note inconsistencies between paragraph text and image

@JScherer-WMF - how does this sound?

This works! Thanks, all!

Change #1186610 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: remove special handling for infobox images

https://gerrit.wikimedia.org/r/1186610

Change #1186642 merged by jenkins-bot:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: remove file name from fallback options

https://gerrit.wikimedia.org/r/1186642

One additional clarifying point:

Detail view:

• don't show paragraph text ever
• if there's a caption, show it
• if no caption, alt text
• if no alt text, nothing

vtoc:

• if paragraph text, show paragraph text.
• if infobox, don't show paragraph
• if gallery, don't show paragraph
• if no paragraph, show caption
• if no caption, show alt
• if no alt, show nothing.

I've captured the new image/text requirements in this task: T404385.

I don't think we have anything else to QA here, I'd recommend that we sign off on both this task as well as T401221 now. We can do extensive QA on some top-ranked articles with the new behavior as part of T404385.

Change #1186646 abandoned by Eric Gardner:

[mediawiki/extensions/ReaderExperiments@master] ImageBrowsing: nearby paragraph as the first fallback option

Reason:

Abandoning as requirements have now changed.

https://gerrit.wikimedia.org/r/1186646

@lwatson, @egardner - we have an AC below which says:

• We perform some review/QA of this using a range of different articles and determine if this is sufficient for the prototype or if we need to do something beyond this

Is this now tracked under a separate ticket and if so, which one? cc @Edtadros, @Etonkovidova

In T402223#11181579, @ovasileva wrote:

@lwatson, @egardner - we have an AC below which says:

• We perform some review/QA of this using a range of different articles and determine if this is sufficient for the prototype or if we need to do something beyond this

Is this now tracked under a separate ticket and if so, which one? cc @Edtadros, @Etonkovidova

I pulled this task out of QA because it's being superseded by https://phabricator.wikimedia.org/T404385.

I think this task can be resolved as-is now (some work was done to the original requirements but now there are new requirements – no point in re-doing the same QA twice).

Test wiki on Patch demo by LWatson-WMF using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/49eab22e2b/w/

Test wiki on Patch demo by LWatson-WMF using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org/wikis/24832aef52/w/

In T402223#11182081, @egardner wrote:

In T402223#11181579, @ovasileva wrote:

@lwatson, @egardner - we have an AC below which says:

• We perform some review/QA of this using a range of different articles and determine if this is sufficient for the prototype or if we need to do something beyond this

Is this now tracked under a separate ticket and if so, which one? cc @Edtadros, @Etonkovidova

I pulled this task out of QA because it's being superseded by https://phabricator.wikimedia.org/T404385.

I think this task can be resolved as-is now (some work was done to the original requirements but now there are new requirements – no point in re-doing the same QA twice).

Sounds good. I will edit the acceptance criteria. @egardner - for future reference, I think the cleanest way to do this would be to:

1. Document the decision in a comment on why the acceptance criteria is changing
2. Edit the task description to change the acceptance criteria based on the decision

Acceptance criteria:

• We perform some review/QA of this using a range of different articles and determine if this is sufficient for the prototype or if we need to do something beyond this

Now moved to T404385: Image Browsing: Update text shown in VTOC and detail view based on new requirements

@egardner - one more question! Which patch demo can be used to confirm AC1?

Hey @ovasileva – here is an up-to-date Patchdemo that shows everything which has been merged into the master branch so far:

https://patchdemo.wmcloud.org/wikis/c434d7c2f2/wiki/Paris

Infobox images that lack captions/alt text currently show no text in the VTOC. Futher refinement will happen as part of T404385.

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/a07111eaec/w/

Test wiki on Patch demo by EGardner (WMF) using patch(es) linked to this task was deleted:

https://patchdemo.wmcloud.org//wikis/a07111eaec/w/