Page MenuHomePhabricator

Change what symbol is prepended to pings on ar.wiki
Closed, ResolvedPublic

Description

This task is about changing what symbol (e.g. @ or ) is prepended to user links created with the Reply Tool's pinging feature.

Background

As @Dyolf77_WMF notes in T252460#6127013, editors at the Arabic Wikipedia are expecting pings created with the Reply Tool's pinging feature to resemble those created with their {{Reply to}} template.

Behavior

  1. Visit a page on ar.wiki like: https://ar.wikipedia.org/wiki/%D9%86%D9%82%D8%A7%D8%B4_%D9%88%D9%8A%D9%83%D9%8A%D8%A8%D9%8A%D8%AF%D9%8A%D8%A7:%D9%85%D8%B4%D8%B1%D9%88%D8%B9_%D8%A3%D8%AF%D9%88%D8%A7%D8%AA_%D8%A7%D9%84%D9%85%D8%AD%D8%A7%D8%AF%D8%AB%D8%A7%D8%AA
  2. Click on a [ reply ] link
  3. In the Reply Tool's visual mode type @
  4. Search for a username by typing some characters
  5. Notice the @ remains in the tool's text input area along with the characters you are typing
  6. Select a username from the username suggestion list by pressing enter

Actual

  1. ❗️Notice the following appears in the Reply Tool's text input: USERNAME@

Desired

  1. ✅ Notice the following appears in the Reply Tool's text input: USERNAME<; this should appear in wikitext as follows: [[User:USERNAME|USERNAME]]<
    • Note: the @ symbol that was present in 3., 4., 5. and 6. should no longer be present

Open questions

  • 1. As @Esanders noted in T250332#6166695, do people find it odd/confusing that the character that is typed to initiate the pinging "flow" is different from the one that ends up being "rendered" within the Tool's text input area and ultimately posted to the page?
    • We will answer the above by sharing a Patchdemo link so @Dyolf77_WMF, and other volunteers at ar.wiki, can experiment with the approach.
  • 2. What – if anything – do we need to do to make sure the "Desired" behavior above does not inhibit editing specific comments T245225?
    • No longer a consideration.

Done

  • All "Open questions" are answered
  • "Desired" behavior is implemented

Event Timeline

DLynch renamed this task from Change what synbol is prepended to pings on ar.wiki to Change what symbol is prepended to pings on ar.wiki.Jul 23 2020, 7:17 PM
Dyolf77_WMF updated the task description. (Show Details)Jul 24 2020, 5:45 PM
Esanders updated the task description. (Show Details)Jul 31 2020, 12:27 PM

Note that all the implementation will be done on T250332. This ticket will just be editing a message on ar.wiki to change it from '@' to '◀'.

Note that all the implementation will be done on T250332. This ticket will just be editing a message on ar.wiki to change it from '@' to '◀'.

Understood – thank you, Ed. Considering this task is about an ar.wiki-specific implementation and T250332 [seems to me to be] is about the underlying infrastructure that enables wikis to customize the pinging syntax, I thought it would be best to invite feedback from ar.wiki on this ticket (see the comment I posted below).

ppelberg added a comment.EditedAug 3 2020, 10:47 PM

@Dyolf77_WMF we've come up with an initial implementation that makes it so when people create a ping in the Reply Tool's visual mode, said ping will have shown before it rather than @ as is currently done.

Question

  • Can you please give this implementation a try, let us know what you think and ask others to do the same? Instructions for how you can try this are listed below; we'd value you sharing these instructions with others at ar.wiki as well.

Testing instructions

  1. Visit this test wiki: http://patchdemo.wmflabs.org/wikis/03cf1da7dd9d63e9e8bb70b558d6086b/w/index.php/Talk:Main_Page
  2. Fill the talk page with whatever content you'd like (right now it's filled with Lorem ipsum)
  3. Open the Reply Tool's visual mode
  4. Try pinging someone by pressing @ and selecting someone from the list that appears
  5. Observe that the username of the person you are wanting to ping will have a before it rather than @ as is currently done.
Dyolf77_WMF moved this task from Backlog to In progress on the User-Dyolf77 board.
  • Can you please give this implementation a try, let us know what you think and ask others to do the same? Instructions for how you can try this are listed below; we'd value you sharing these instructions with others at ar.wiki as well.

Thanks,

  • (When I added, <div dir="rtl">, see the second section, to obtain a RTL display, all the indentations are not working).

Unfortunately indentation in MediaWiki doesn’t depend on the dir attribute, but on the mw-content-{ltr,rtl} classes. <div dir="rtl" class="mw-content-rtl"> works.

  • My observations:
    • Usernames have this symbol instead of the @ when pinging them from the list;
    • This symbol is flipped. In a RTL text, is what we expect;
    • (When I added, <div dir="rtl">, see the second section, to obtain a RTL display, all the indentations are not working).

The demo wiki has English as the content language, so we weren't able to set that up properly, but it should work correctly on the real wiki. We should try this one more time on the Beta Cluster (https://ar.wikipedia.beta.wmflabs.org/) after the required patches are merged.

@Dyolf77_WMF we've come up with an initial implementation that makes it so when people create a ping in the Reply Tool's visual mode, said ping will have shown before it rather than @ as is currently done.

Question

  • Can you please give this implementation a try, let us know what you think and ask others to do the same? Instructions for how you can try this are listed below; we'd value you sharing these instructions with others at ar.wiki as well.

Update
Yesterday, @Dyolf77_WMF confirmed pinging is working as expected on ar.wiki.

Next steps
Per T258743#6366729, once T250332 is resolved, we will test this once more on the Beta Cluster (https://ar.wikipedia.beta.wmflabs.org/).

This being tested on https://ar.wikipedia.beta.wmflabs.org/ and subsequently "signed off on" depends on T250332 being reviewed and deployed.

ppelberg reassigned this task from Dyolf77_WMF to Esanders.Aug 14 2020, 10:55 PM

This should work correctly on https://ar.wikipedia.beta.wmflabs.org/ now.

Tested and working correctly, thanks! But I noticed a </nowiki> tag in the edit, is that expected?

I did not expect that.

But I can explain why: Arabic (unlike English) uses "link prefixes", so if you wrote that without the <nowiki/>, the character would be displayed as part of a link instead of as plain text before it. This is similar to how [[cat]]s results in "cats", except it applies to text before the link, rather than after. I didn't realize that link prefixes apply to any character, unlike link suffixes, which only apply to actual letters/numbers and not other symbols.

The result renders correctly, but the nowiki in wikitext certainly can be annoying. I'm not sure how to fix it though. Even ZWNJ and other zero-width characters are pulled into the link. We might have to actually give up on this approach and use the mention template instead, or live with the nowikis.

Thanks @matmarex I appreciate your explanation!

The result renders correctly, but the nowiki in wikitext certainly can be annoying. I'm not sure how to fix it though. Even ZWNJ and other zero-width characters are pulled into the link. We might have to actually give up on this approach and use the mention template instead, or live with the nowikis.

@ppelberg @Whatamidoing-WMF should I ask the community in ar.wiki?

Would this problem be solved by adding a space between the prefix and the arrow?

Would this problem be solved by adding a space between the prefix and the arrow?

It would, but I assume that's also not ideal.

You can test this by setting the message to ◀&nbsp; (a normal space would get removed when saving the message page, plus you probably want a non-breaking space anyway)

You can test this by setting the message to ◀&nbsp; (a normal space would get removed when saving the message page, plus you probably want a non-breaking space anyway)

I think the 'non-breaking space' should be in the left of the arrow &nbsp;◀.

It should appear on the left but I wrote it in LTR here on Phab so it can be copy-pasted into the wiki. In RTL it actually renders as ;nbsp&◀ in the editor.

It should appear on the left but I wrote it in LTR here on Phab so it can be copy-pasted into the wiki. In RTL it actually renders as ;nbsp&◀ in the editor.

Thanks.

You can test this by setting the message to ◀&nbsp; (a normal space would get removed when saving the message page, plus you probably want a non-breaking space anyway)

@Dyolf77_WMF can you, or someone else who has the rights to do so, please change the interface message for the @ mention prefix to ◀&nbsp; on https://ar.wikipedia.beta.wmflabs.org and after doing so, asking the community at ar.wiki what they think of this approach?

Asking volunteers at ar.wik
Habib, when you ask volunteers at ar.wiki what they think about the @ mention prefix interface message being changed to ◀&nbsp;, we will be specifically curious to know the following:

  • 1. Is the space between the black left-pointing triangle () and the person being mentioned's username acceptable? Is this behavior preferable to the </nowiki> tag you noted in T258743#6387675 ?
  • 2. Do they observe anything unexpected when doing the following?
    • 1. Write a comment that contains a ping in the Reply Tool's visual mode
    • 2. Switch to the Reply Tool's source mode
    • 3. Switch back to the Reply Tool's visual mode
    • 4. Publish the comment you drafted in "Step 1"
    • 5. Inspect the diff Step 4" will produce

Hi @ppelberg

@Dyolf77_WMF can you, or someone else who has the rights to do so, please change the interface message for the @ mention prefix to ◀&nbsp; on https://ar.wikipedia.beta.wmflabs.org and after doing so, asking the community at ar.wiki what they think of this approach?

The symbol has been changed but the </nowiki> is still there.

Hi @ppelberg

@Dyolf77_WMF can you, or someone else who has the rights to do so, please change the interface message for the @ mention prefix to ◀&nbsp; on https://ar.wikipedia.beta.wmflabs.org and after doing so, asking the community at ar.wiki what they think of this approach?

The symbol has been changed but the </nowiki> is still there.

@Dyolf77_WMF, thank you for configuring the interface message and alerting us this issue still persists. We'll see if we can correct this so the <nowiki/> syntax is no longer inserted into the comment.

I think the non-breaking space is still caught in the link prefix, as I explained earlier (T258743#6388738). &#32; to generate a normal space might work (but then, you might get line breaks between the and the username, which seems bad).

Dyolf77_WMF added a comment.EditedAug 26 2020, 10:01 PM

When a : is added to the prefix, the </nowiki> tag disappeared from the text.
The space between the symbol and the username is not a big issue, but, in my opinion, the additional : can be annoying/disturbing for the community.
Update: The </nowiki> tag disappears also with only the symbol+: ( this form ).

Esanders added a comment.EditedAug 31 2020, 6:11 PM

I think the root cause of this is that Arabic, and link prefixes in general, are incorrectly configured in MediaWiki.

In MediaWiki you can have:

  • a link trail: [[hold]]en = [[hold|holden]]
  • and a link prefix: be[[holden]] = [[holden|beholden]]`.

The characters which can be used for this are configured per-language in MediaWiki core.

All configs fall back to English, so I will refer to that as "default":

  • Suffixes (link trails) are enabled by default, and match letters a-z ($linkTrail = '/^([a-z]+)(.*)$/sD';)
    • Arabic extends this pattern to include Arabic letters: [a-zء-ي]. This is sensible.
  • Prefixes are disabled by default, but the default pattern when they are enabled is any character A-Z and any character after 0x80 in the Unicode table.
    • This is not sensible as it only excludes non-letters that appear early in the Unicode table, this is basically classic ASCII punctuation (which is why '@' can be used), but doesn't exclude any non-letters from other languages, or more esoteric glyphs such as the black triangles.
    • The following languages have prefixes enabled: ar, cu, cv, hy, is, ka, kaa, lbe, ln, mzn, pnb, uk, uz

Arabic is configured to enable prefixes, but uses the default pattern.

I think any language that enables link prefixes should define a sensible pattern for what should get matched (e.g. letters in the local alphabet).

The problem with any changes to these configs is that they will potentially break any existing content that relied on this "broken" functionality.

cscott added a subscriber: cscott.Aug 31 2020, 7:26 PM

Our recommendation is to forward this issue to Language-Team (Language-2020-October-December) to address. The "proper" solution is (in our opinion) is:

  1. Delegate the Language Team to interact w/ the local wiki communities on each of the <13 wikis which have link prefixes enabled but are currently using the (bogus) default link prefix, to have them either turn off link prefixes or else define a more appropriate prefix charset for their language.
  2. Once zero wikis are using the default link prefix setting, patch core to change the default to match nothing; this would force any future wiki which wants to use link prefixes to define a sane prefix.
  3. Use RELEASE-NOTES and a note in the DefaultSettings.php to explain which characters should definitely *not* be included in link prefix (ie, defining a link prefix that matches &nbsp; is a bad idea).

It seems tempting to just define the default linkprefix to \L, but (a) this didn't exist when linkprefix was invented (see db9c4cb3cfbf52a79f89e801f461ad17fbdb1975, 3f026e7518d95048d512b71b5d9898a2df5d29d2, and 6aa81d268b31e58d74af875e46281fdddf90c5f4 for some history), and (b) turning on unicode processing with the u regexp flag has bad side-effects in PHP. Not only does it significantly slow down regexp processing (and link handling is an important component of parsing), *any* invalid UTF-8 characters in the subject will cause preg_* functions to match nothing (see https://www.php.net/manual/en/reference.pcre.pattern.modifiers.php). Although wikitext is *supposed* to be UTF-8 clean, bad characters sometimes sneak in (eg I just fixed T237467: Invariant failed: Bad UTF-8 (full string verification) today), and it would be very surprising for (say) all link processing on the page to fail because a bad character snuck into a template expansion.

@edsanders suggested that "inherit from suffix" would be a better default -- but historically we've been terrible at defining proper link prefixes (see db9c4cb3cfbf52a79f89e801f461ad17fbdb1975) and so I think it's better to "do no harm" by effectively not having a link prefix at all if the prefix is not customized, rather than to set an arbitrary prefix which is probably no more correct that our previous default was.

ppelberg added a comment.EditedAug 31 2020, 7:48 PM

Nice work, @Esanders. Resolving the root cause of this issue – identified in T258743#6424685 and which @cscott elaborated on in T258743#6424887 – will happen in this task: T261707.


In the meantime, @Dyolf77_WMF, below are four ways in which the software could enable people at ar.wiki to have access to the functionality this task was intended to introduce. [i] Habib, two resulting questions for you:

  • 1. Can you confirm the approaches below make sense to you?
  • 2. Can you indicate which of the approaches you think we should implement? Of course, if you think it would be helpful to get input from others at ar.wiki, please go ahead and ask people on the wiki!

APPROACHES
Below are four different approaches for how to avoid the <nowiki/> issue identified in T258743#6387675 . Note: implementing any of the approaches below would be temporary while the root cause is being fixed.

  • Approach #1: use a character for the ping prefix that that is within the 0x80 ASCII range (e.g. > or <)
  • Approach #2: incorporate the ◀️ into the ping link
    • Implementation: pings made with the Reply Tool at ar.wiki would look like this: [[User:Dyolf77_WMF|◀Dyolf77_WMF]]
    • Consideration: this would mean the following functionality would be lost:
      • 1. Open the Reply Tool's visual mode
      • 2. Create a ping for @Dyolf77_WMF
      • 3. Press backspace to delete the ping you created in "Step 2"
      • 4. Notice the username completion list would NOT be shown (it currently is shown in this scenario) [ii]
  • Approach #3: use a normal space
    • Consideration: a ping included in the middle of a comment could get broken up (read: wrap across multiple lines).
    • Editing Team comments: we think this would be the least optimal approach of the four mentioned here.
  • Approach #4: do nothing (read: continue using the @ symbol as the ping prefix until the root cause is addressed)

i. At ar.wiki, make it so the Reply Tool outputs the following when someone creates a ping in the tool's visual mode: [[User:USERNAME|USERNAME]]◀
ii.

Although wikitext is *supposed* to be UTF-8 clean, bad characters sometimes sneak in (eg I just fixed T237467: Invariant failed: Bad UTF-8 (full string verification) today), and it would be very surprising for (say) all link processing on the page to fail because a bad character snuck into a template expansion.

This is tangential to the main discussion, but, I don't think that would be a bad thing necessarily in that it would flag problems by crashing on the page.

ppelberg updated the task description. (Show Details)Aug 31 2020, 9:59 PM

In the meantime, @Dyolf77_WMF, below are four ways in which the software could enable people at ar.wiki to have access to the functionality this task was intended to introduce. [i] Habib, two resulting questions for you:

  • 1. Can you confirm the approaches below make sense to you?
  • 2. Can you indicate which of the approaches you think we should implement? Of course, if you think it would be helpful to get input from others at ar.wiki, please go ahead and ask people on the wiki!

@Dyolf77_WMF and I talked about this offline and arrived at the following decision: Implement "Approach #1: in the short-term, perpend pings at ar.wiki with the < character. The task description's "Behavior" section has been updated to include this new approach.

Once the root cause is fixed, we'll re-implement what this task originally asked for: [[User:USERNAME|USERNAME]]◀. This work will happen in this task: T261709.

Root cause filed as T261750.

Using > means there is no more engineering work to do. Once @Dyolf77_WMF has verified this works on Arabic beta wiki we can call this done.

DLynch added a comment.Sep 1 2020, 3:03 PM

Is > going to cause any potential weird HTML issues? (Or complaints about weird diffs if it's escaped out to &lt;?)

Using > means there is no more engineering work to do. Once @Dyolf77_WMF has verified this works on Arabic beta wiki we can call this done.

The new < in Arabic beta wiki, is NOT causing any visible issue:

https://ar.wikipedia.beta.wmflabs.org/w/index.php?title=%D9%86%D9%82%D8%A7%D8%B4_%D8%A7%D9%84%D9%85%D8%B3%D8%AA%D8%AE%D8%AF%D9%85:Dyolf77&diff=prev&oldid=1328

Is > going to cause any potential weird HTML issues? (Or complaints about weird diffs if it's escaped out to &lt;?)

It looks like Parsoid escapes text that would look like a HTML tag with <nowiki>…</nowiki>: https://ar.wikipedia.beta.wmflabs.org/w/index.php?title=نقاش_المستخدم:Dyolf77&diff=1333&oldid=1330. I think this is fine (especially since it's only Latin script letters that can cause the problem, as HTML tags are not in Arabic, so this would be very rare).

ppelberg closed this task as Resolved.Sep 1 2020, 11:51 PM

Using > means there is no more engineering work to do. Once @Dyolf77_WMF has verified this works on Arabic beta wiki we can call this done.

The new < in Arabic beta wiki, is NOT causing any visible issue:

https://ar.wikipedia.beta.wmflabs.org/w/index.php?title=%D9%86%D9%82%D8%A7%D8%B4_%D8%A7%D9%84%D9%85%D8%B3%D8%AA%D8%AE%D8%AF%D9%85:Dyolf77&diff=prev&oldid=1328

This is great news! We appreciate how patient and experimental you've been, Habib.

I'm going to consider this task "Resolved." Please let us know if you, or anyone else, notices issues.

Restricted Application added a project: User-Ryasmeen. · View Herald TranscriptSep 1 2020, 11:51 PM

Is > going to cause any potential weird HTML issues? (Or complaints about weird diffs if it's escaped out to &lt;?)

I'm glad you raised this prospect @DLynch.

It looks like Parsoid escapes text that would look like a HTML tag with <nowiki>…</nowiki>: https://ar.wikipedia.beta.wmflabs.org/w/index.php?title=نقاش_المستخدم:Dyolf77&diff=1333&oldid=1330. I think this is fine (especially since it's only Latin script letters that can cause the problem, as HTML tags are not in Arabic, so this would be very rare).

Nice to know how the software behaves in this case, @matmarex.