Page MenuHomePhabricator

Captions Panel alignment and overlapping issues in RTL languages
Closed, ResolvedPublicBUG REPORT

Description

Steps to Reproduce:
The Captions Panel presents itself like this (exemplified on Hebrew):

Screen Shot 2019-04-16 at 9.29.26 AM.png (307×1 px, 27 KB)

There are mis-alignments and overlaps that need to be tackled

Expected Result

With an RTL language the primary language of the interface, the panel should look like this:

captions.png (176×836 px, 15 KB)

(layout comments in dark pink)

QA Results

ACStatusDetails
1T222867#5856679

Event Timeline

Ramsey-WMF moved this task from Untriaged to Next up on the Multimedia board.

There is a lot I don't know about the way that Mediawiki handles RTL language mode. It seems like the first problem here – the fact that the Captions panel remains aligned to the left side of the screen, even when most other things align to the right – is visible even when Javascript is disabled:

Screen Shot 2019-05-23 at 3.38.26 PM.png (1×1 px, 612 KB)

This makes me wonder if the problem has to do with some of the new server-side OOUI classes that were recently added to the PHP version of the library for use in Mediainfo? Is it possible that one of the new server-side UI widgets doesn't implement RTL switching correctly? Pinging @Cparle since he was involved in getting those additional server-side components added for the tabs layout.

The server-generated markup for the file page looks like this before any JS gets loaded (I have removed some data-attributes for clarity). Phabricator makes this kind of painful, but look at div#ooui-php-4 down below:

<!-- top-level tabs container -->
<div class="wbmi-tabs-container oo-ui-layout oo-ui-panelLayout">

    <!-- top level div for all tabs -->
    <div id="ooui-php-1"
	 class="wbmi-tabs oo-ui-layout oo-ui-menuLayout
		 oo-ui-menuLayout-static oo-ui-menuLayout-top oo-ui-menuLayout-showMenu
		 oo-ui-indexLayout">

        <!-- next comes the JS menu elements, but they are hidden when JS is disabled so I'm leaving out for simplicity -->

        <!-- next comes the main content -->
	<div class="oo-ui-menuLayout-content">
		<div id="ooui-php-3" class="oo-ui-layout oo-ui-panelLayout oo-ui-stackLayout oo-ui-indexLayout-stackLayout">

                    <!-- here is the content of the first tab -->
                    <div role="tabpanel" id="ooui-php-4" class="wbmi-tab oo-ui-layout oo-ui-panelLayout oo-ui-panelLayout-scrollable oo-ui-tabPanelLayout">
                        <h2 class="wbmi-captions-header">כיתובים</h2>

                        <!-- **I think this is our culprit: note the direction of "LTR" and related CSS class, even though the rest of the page is RTL** -->
			<div id="mw-imagepage-content" dir="ltr" class="mw-content-ltr" lang="en">
				<div class="mw-parser-output">
					<div data-caption-languages="he"
						 class="wbmi-entityview-captionsPanel oo-ui-layout
						 oo-ui-panelLayout oo-ui-panelLayout-framed">

						<h3 class="wbmi-entityview-captions-header">כיתובים</h3>

						<div class="wbmi-entityview-caption
						wbmi-entityview-showLabel oo-ui-layout
						oo-ui-horizontalLayout">
							<label aria-disabled="false"
								   dir="rtl"
								   class="wbmi-language-label oo-ui-widget
								   oo-ui-widget-enabled
								   oo-ui-labelElement-label
								   oo-ui-labelElement oo-ui-labelWidget"
								   lang="he">
								עברית
							</label>

							<div dir="rtl"
								 class="wbmi-caption-value
								 wbmi-entityview-emptyCaption"
								 lang="he">
								נא להוסיף משפט שמסביר מה הקובץ מייצג
							</div>
						</div>
					</div>
				</div>
			</div>
		</div>

    <!-- remainder omitted for brevity -->    
</div>

Where are you testing this?

Are you using debug mode? Last time I checked, RTL was more broken in debug mode.

This is running in a local development environment on my machine. I just switched the language to Hebrew using the menu at the top of the page. But adding “uselang=he” to the URL creates the same results. Haven’t been looking at debug mode yet.

That's because the captions panel is inside <div id="mw-imagepage-content">, which has lang="en", dir="ltr" and class="mw-content-ltr".
It makes sense that that one has LTR, because it indicates the start of the "content" of the page (not the interface), which on that page, on Commons, is English.

I'm not sure what the best approach is. Probably pull it out of that node (if at all possible) in the Hooks.php functions that deal with that, and append it before the content block?
I don't know how feasible that is, and what parser cache effects it would have - would have to investigate. Maybe other solutions are possible/better.

I'm willing to give that a try and see what the results are locally. Would pulling Captions out of the content block mean that it no longer gets included in the parser cache?

Most of that code is somewhere in WikibaseMediaInfoHooks::doBeforePageDisplay (and the various methods called from there)
That method is run via a hook shortly before everything gets output.
Basically, the content is already there (from parser cache, or generated at that time) and the code in that method will extract it from that location, do some things with it (e.g. add tabs) and move some bits and pieces around.
All of that happens after the parser cache, so should probably be fine.

I suspect that we'd only need to alter the string replace code that follows // inject captions into tab1, so that it adds captions elsewhere.
(but I've not actually tested & validated any of what I just said, so all of this may be misinformation :D)

Change 512435 had a related patch set uploaded (by Eric Gardner; owner: Eric Gardner):
[mediawiki/extensions/WikibaseMediaInfo@master] Ensure that Captions panel is right-aligned in RTL languages

https://gerrit.wikimedia.org/r/512435

Thanks @matthiasmullie for putting me on the right track. I was able to do exactly what you suggested to prepend the captions HTML to the rest of the tab1 contents, so that it comes before #mw-imagepage-content and #mw-parser-output. As far as I can tell, that solves the initial problem here – the Captions panel is now correctly oriented in an RTL page. Here's what it looks like in my local dev environment:

Screen Shot 2019-05-24 at 1.44.52 PM.png (1×2 px, 1 MB)

CSS and JS seem to work fine without any other changes.
Patch for this is up here: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseMediaInfo/+/512435

This does not fix the other issues of this panel yet, but I will investigate those next.

Here is a follow-up question about the layout of the Captions panel contents themselves, when both RTL and LTR languages are present.

@Volker_E, in the initial ticket description you included a mockup for the Captions panel where each language-row gets oriented to the left or the right based on the natural behavior for the language in question. A panel with two language-rows, one in Hebrew and the other in English, will always show the "English" label on the left and the "Hebrew" label on the right, regardless of whether the user is viewing the overall page in RTL or LTR orientation.

Are we sure we prefer that behavior to something like this (see below)? The idea here is that the label would always come first in the reading order of the overall page, based on whether the user is using an RTL or LTR UI. Text inside labels would flow in the natural direction for the language, of course. If the user set the UI language to Hebrew, they'd see something like the screenshots below; an English user would see the reverse.

Screen Shot 2019-05-24 at 1.59.50 PM.png (352×1 px, 37 KB)

Screen Shot 2019-05-24 at 2.00.31 PM.png (482×1 px, 55 KB)

If this is *not* what we want I will follow the initial guidelines above, just thought I would raise this as another option.

@egardner I'm pretty sure that the proposed treatment is just exchanging the burden from RTL readers to LTR users. Does this interface with English language on the right really speak to you? Would it speak to a Hebrew user who is also capable of English? I don't think it makes more sense, even though the proposed treatment in the task description might look noisier in the beginning. Would be interested to hear opinion of @Amire80 here as well…

I think that there is something to be said for the visual tidiness of this approach, and that using a consistent placement of label elements might allow users to figure out what is going on here more quickly. I find this approach to be more intuitive, personally.

But I defer to the opinions of folks who have been working on this problem for a lot longer than I have and/or who actually read in some of these languages.

Change 512435 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Ensure that Captions panel is right-aligned in RTL languages

https://gerrit.wikimedia.org/r/512435

so ... what's the status here? We have a merged patch, but it's still in doing ... @egardner ?

@egardner: Could you reply to the previous comment, please? Thanks!

@Aklapper part of this issue has been resolved with the merger of this patch: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/WikibaseMediaInfo/+/512435. That took care of the problem where the entire Captions panel remained stuck on the left side when it should flip along with the text orientation (LTR or RTL).

But I don't think there was ever a final verdict on how the contents of these panels should behave in RTL mode (or whenever both LTR and RTL languages are present within the same panel). @Volker_E and @PDrouin-WMF if you have any further thoughts let me know and I can do another pass on this.

Since this issue has fallen to the wayside for a while, I want to put an updated summary here so we can see everything in a single comment. My goal is to get final clarification on what the desired behavior here is.

Problem: When we try to handle both RTL and LTR languages inside of the Captions panel, there are a lot of visual inconsistencies. Let's figure out which, if any, should be addressed.

Examples: All of the screenshots below assume the following situation – a user with an RTL UI (Hebrew, in this case), needs to read or interact with captions in multiple languages that are both RTL and LTR.

  1. Placeholder text aligns differently per language (nothing lines up visually)
    Screen Shot 2019-09-10 at 8.47.57 AM.png (164×821 px, 17 KB)
  2. Input elements and dropdown controls line up differently (though some elements – trash icon, down arrow) are the same for both
    Screen Shot 2019-09-10 at 8.48.07 AM.png (230×827 px, 23 KB)
  3. Text that the user types goes in different directions based on language
    Screen Shot 2019-09-10 at 8.48.19 AM.png (235×842 px, 18 KB)
  4. Saved text in different languages; nothing lines up
    Screen Shot 2019-09-10 at 8.48.34 AM.png (177×828 px, 13 KB)

My personal opinion (as someone who does not read RTL languages and will defer to folks who do) is that we could fix a lot of visual inconsistencies by aligning the English text to the right (without otherwise changing how it displays). This will ensure that text doesn't jump all over the page to create a generally chaotic experience.

The main example of the behavior that I'm advocating for comes from the way the page titles are handled – in this case, the title is in English but it still gets aligned to the right side of the page:

Screen Shot 2019-09-10 at 8.57.07 AM.png (1×1 px, 439 KB)

I think we could apply similar visual rules inside the captions panel, but I defer to the design team here.

I'm having a bit of a deja vu here because I think it was discussed already on some other task. But I don't mind repeating ;)

These screenshots from a previous comment on this task look mostly good:

Screen Shot 2019-05-24 at 1.59.50 PM.png (352×1 px, 37 KB)

Screen Shot 2019-05-24 at 2.00.31 PM.png (482×1 px, 55 KB)

The appearance of the names of the languages is perfect. They are all aligned to the same side, and it's both nice and readable. Each language name must also be in an HTML element with appropriate lang and dir attributes, but the block-level text-align must all be forced to the same side in all the language names: right in Hebrew UI, left in English UI.

The appearance of the description should probably be adjusted, however. The appropriate lang and dir attributes must be applied, and the alignment should follow the direction: left for English lines, right for Hebrew lines. The reason for this is that they can be editable, and when you edit them, the direction should be the same as the alignment, otherwise the letters appear to go backwards. It's readable, but looks a bit awkward.

Hope it helps.

I'm not sure what outcome is really better in a mixed language interface. If I were in an English first interface, I'd still not assume to have Hebrew necessarily aligned with English just to make it look better.

Another thing to consider is, that we rely on certain positioning in OOUI, which – in the proposal from previous comment in Amire's last message – might have to be overriden in a maintenance burdensome way.

I'm not sure what outcome is really better in a mixed language interface. If I were in an English first interface, I'd still not assume to have Hebrew necessarily aligned with English just to make it look better.

We thought about it back in 2012 or so when we were designing ULS, and we came to the conclusion to align all the language names to the same side.

Heads-up - we're going to start addressing some of these issues soon, but there's a refactoring patch that has to go out first (https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseMediaInfo/+/528825/).

Heads-up - we're going to start addressing some of these issues soon, but there's a refactoring patch that has to go out first (https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/WikibaseMediaInfo/+/528825/).

That patch got merged - this is no longer blocked.

I'd like to wrap this one up and after reading the comments it seems we can move forward with the following UI in RTL language mode:

captions in edit mode.png (241×897 px, 24 KB)

captions.png (176×836 px, 15 KB)

@Amire80, can you please confirm that this is correct? If so, and if no one else has feedback, I'll move forward on this. Thanks!

I'd like to wrap this one up and after reading the comments it seems we can move forward with the following UI in RTL language mode:

captions in edit mode.png (241×897 px, 24 KB)

captions.png (176×836 px, 15 KB)

@Amire80, can you please confirm that this is correct? If so, and if no one else has feedback, I'll move forward on this. Thanks!

This looks good!

Change 564757 had a related patch set uploaded (by Anne Tomasevich; owner: Anne Tomasevich):
[mediawiki/extensions/WikibaseMediaInfo@master] Use UI language direction for caption language labels

https://gerrit.wikimedia.org/r/564757

Change 564757 merged by jenkins-bot:
[mediawiki/extensions/WikibaseMediaInfo@master] Use UI language direction for caption language labels

https://gerrit.wikimedia.org/r/564757

This should be testable on Beta.

Edtadros subscribed.

Test Result

Status:
OS: macOS Catalina
Browser: Chrome
Device: MBP

Test Artifact(s):

QA Steps

❌ AC1: With an RTL language the primary language of the interface, the panel should look like this:

T222867-SDC-Captions-Panel-RTL.png (307×1 px, 20 KB)

Used the following: https://commons.wikimedia.beta.wmflabs.org/wiki/File:Ffgg_.jpg?uselang=he
The entities in the image do not align as shown in the expected result above.

Screen Shot 2020-02-06 at 8.13.27 AM.png (322×865 px, 20 KB)

Based on discussions that Amir and Anne had, the expected result shifted over time but the main task description wasn't updated. I've updated that, and based on that it looks like this task does pass QA

Tested on prod and works per the updated desired spec.