VisualEditor: Provide a character insertion tool
Closed, ResolvedPublic

Description

The VisualEditor doesn't currently have a mechanism for inserting special characters - while a lot of them are unnecessary (~~~~ for example), things like en and emdashes can be quite important for content.


Version: unspecified
Severity: enhancement
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=11130

bzimport set Reference to bz50296.
Ironholds created this task.Via LegacyJun 27 2013, 4:06 PM
Jdforrester-WMF added a comment.Via ConduitJun 27 2013, 5:58 PM

(In reply to comment #0)

The VisualEditor doesn't currently have a mechanism for inserting special
characters - while a lot of them are unnecessary (~~~~ for example), things
like en and emdashes can be quite important for content.

Character insertion is, in general, the responsibility of your browser/operating system (same as spell check, IME and other general editing tools). Where your local OS fails, we have a 'standard' of providing general tools to let you get OS-level tools (most notably, ULS), rather than one-off "insert a bunch of random characters" micro-tools.

MediaWiki has given people tools to insert random characters primarily because they do special things, and we've given up on users being able to magically know them (like ~~~~ or {{DEFAULTSORT:}}). They're all examples of system design failure. The fact that we have extended the metaphor to give "helpful" characters that users can trivially type themselves isn't a great reason to create it from scratch.

I'm not dead-set against this, but it feels like a clunky failure. :-(

Ironholds added a comment.Via ConduitJun 27 2013, 6:03 PM

Well, I'm sat on my windows machine here and can't see any way to type an emdash, for example :/

Rschen7754 added a comment.Via ConduitJul 2 2013, 3:47 AM

Nonbreaking spaces would be helpful as well.

He7d3r added a comment.Via ConduitJul 2 2013, 2:05 PM

There was a similar request on Portuguese Wikipedia[1]. A user wanted to use the kind of tool we see in the screenshot [2].

[1] https://pt.wikipedia.org/wiki/WP:Editor_Visual/Coment%C3%A1rios?oldid=36280987&uselang=en#Adi.C3.A7.C3.A3o_de_caracteres_especial_no_Visual_editor
[2] http://postimg.org/image/mwbyn96gn/

gpaumier added a comment.Via ConduitJul 19 2013, 7:50 PM
  • Bug 51120 has been marked as a duplicate of this bug. ***
TheDJ added a comment.Via ConduitJul 19 2013, 7:59 PM

"Character insertion is, in general, the responsibility of your browser/operating system"

Yes and no. Yes, your computer is capable of inserting loads of characters. No, it is not always reachable.

At least, I know of no operating system + browser combination, where I can put my cursor in a text area and bring up a symbol picker from the menu or a toolbar. (Unlike spellchecking).

TheDJ added a comment.Via ConduitJul 19 2013, 8:01 PM

BTW, if we ever do get another symbol picker, please make it share it's table with WikiEditor, limiting the places where we need to maintain these chars. (We already have 3 [WikiEditor extension, MediaWiki:Edittools, MediaWiki:Edittools.js])

gpaumier added a comment.Via ConduitJul 19 2013, 8:03 PM

Agreed with comment 8. The special characters insertion tool in WikiEditor is very convenient when editing articles about topics that involve other scripts or diacritics. Not providing such a tool in VisualEditor would seriously reduce VE's usefulness, and would certainly provide a compelling reason to use the source editor much more often than otherwise necessary.

bzimport added a comment.Via ConduitJul 20 2013, 9:47 PM

eleriygarregwen wrote:

Another example of a language whose users make heavy use of the character insertion tool underneath the old edit window is Welsh (cy). Some of us work with keyboards which have been specially adapted to easily type the diacritics needed (including ŵ and ŷ) but not all hardware devices have this software loaded. This means that the VisualEditor is very difficult to use for some users, making training sessions for new users (some with, some without the diacritics software) difficult to conduct using VisualEditor.

Lydia_Pintscher added a comment.Via ConduitJul 28 2013, 1:49 PM

This is among the most important missing features for dewp.

bzimport added a comment.Via ConduitJul 29 2013, 10:17 PM

eleriygarregwen wrote:

To make life easy on the Welsh Wikipedia, we have added a short character set consisting only of the most commonly used special characters immediately below the edit window. We have also added a more comprehensive set of special characters further down below the edit window. I think it would be useful to add the full character set to VisualEditor as standard, but also to allow each Wikipedia to add its own special character set via an optional message.

Elitre added a comment.Via ConduitAug 23 2013, 11:43 AM

Lloffiwr, can you link to this workaround? I tried enabling VE on your wiki but that was not enough, should I also enable some gadget maybe?
James, until this or other IME issues are not solved, isn't there a way to let these wikis use their very own edit tools, i.e. http://pms.wikipedia.org/wiki/MediaWiki:Edittools ?

bzimport added a comment.Via ConduitAug 27 2013, 10:16 PM

eleriygarregwen wrote:

"Lloffiwr, can you link to this workaround?" - Sorry I don't know what you are asking me to do.

With regard to using our own edit tools, that would be very good. It is very simple to just click on one of a number of characters sitting on a toolbar right next to the editor. You don't have to remember key combinations, there is no need to look up a help page and learn how to insert characters. The character toolbar's purpose in the normal editor is obvious and it is ready to use for all users (you don't have to enable a special keyboard as you would with the ULS).

By the way, in the "save" dialog of the VisualEditor on CYWP, we see an image of our short character toolbar, but have no way of actually using it! I, an experienced editor, was very confused by this; the new editors for whom this tool is designed are likely to be completely floored by it.

Elitre added a comment.Via ConduitAug 28 2013, 11:13 AM

Sorry Lloffiwr, my bad: I thought that Welshp WP had somehow made that set of characters already available with VisualEditor as well.

Thryduulf added a comment.Via ConduitSep 20 2013, 2:14 PM

Another request today for [[MediaWiki:Edittools]] or an equivalent to be available in VE.

I second comment 8 and comment 10 - this is one of the features of the source editor that editors really miss.

bzimport added a comment.Via ConduitSep 20 2013, 3:22 PM

dgerard wrote:

Yep. When you need to add an accented character, you need to add an accented character. This is a quite definitely missing facility.

Risker added a comment.Via ConduitSep 23 2013, 6:21 AM

I need to point out the irony of developing an editing interface with the good faith but technically challenged user in mind, and then expecting them to know how to use their browser/operating system to insert an emdash (a character not present on the standard English keyboard, but used over a million times on English Wikipedia). Given the target group for this extension, this isn't a low-priority enhancement, it is core functionality. In fact, the existing editorial base uses the edit tools almost exclusively over their browser/operating system, and I suspect many of them (including me) have no idea how to do use the latter.

Thryduulf added a comment.Via ConduitSep 24 2013, 10:21 AM

Further to comment 18 and comment 19 a big use case of the character insertion toolbar is to insert characters one does not frequently use.

Utilities like an external character map are very inefficient because it disrupts the work flow, requires you to find the character you want, and copy and paste the character you need. It also replaces whatever else you had in the clipboard previously that you may still need.

Dead keys are useful only if you know that they exist, know how to set them up (they are not enabled by default on most US and UK systems at least), and know the combinations required to input the character you want. I have a much better memory for things like that than many people, and while I know ones like é, Ŵ, ß, ° and ö, characters like ř, Đ and ố I don't. As far as I know, characters in other scripts like д, ζ, ب ,ש, are not producible using deadkey combinations.

Input methods are a very heavy weight solution to inputting characters not found natively on one's keyboard. They always require setting up in advance for each script (and some script/language combinations I think) and learning how to produce each desired character. I only use this method for IPA because I did a lot of work with pronunciations at Wiktionary and that required learning that to produce /ˌbʌɡˈzɪlə/ I need to type /%bVg"zIl@/ - hardly something we can expect a casual editor to learn.

One of the primary goals of VisualEditor is to reduce the barrier to contributing to Wikimedia projects, not having a character insertion tool is rather contradictory to this.

bzimport added a comment.Via ConduitSep 24 2013, 10:25 AM

dgerard wrote:

The other problem is that with VE having no source mode as yet, I can't even resort to HTML entities.

bzimport added a comment.Via ConduitOct 1 2013, 6:38 PM

pearley wrote:

Czech editors have compiled a list of characters they find are vital for their day-to-day editing. From https://cs.wikipedia.org/wiki/Diskuse_k_Wikipedii:VisualEditor

Output Name Entity Decimal

	nobreak space	                         	 
	thin space	                         

„ curly quotation marks begin (99) „ „
“ curly quotation marks end (66) “ “
‚ simple curly quotation marks begin ‚ ‚
‘ simple curly quotation marks end ‘ ‘
– n-dash – –
× times × ×

In particular, both kinds of quotation marks (we are very particular about those, like the French about theirs) could be insertable either one by one or in pairs, like so: „“ and ‚‘.
In general, easy access to inserting those characters would be a very desirable feature for Czech editors – and even better would be the MS Word functionality that automatically converts minuses into n-dashes and "English quotation marks" into „Czech quotation marks“. Best, Littledogboy (diskuse) 1. 10. 2013, 11:35 (UTC)
Important: Although I've listed their codes, we usually enter n-dash and curly quotation marks NOT as html entities, but simply like so „–“ (Alt-0132, Alt-0150, Alt-0147 – yes, I am a robot). Littledogboy (diskuse) 1. 10. 2013, 11:51 (UTC)

Jdforrester-WMF added a comment.Via ConduitOct 9 2013, 2:39 PM
  • Bug 55496 has been marked as a duplicate of this bug. ***
bzimport added a comment.Via ConduitOct 28 2013, 3:10 AM

lebo.beland wrote:

I didn't read the whole thread, but I suggest to look at including Mediawiki:Edittools in the Visual Editor. For many projects, this include the special characters and useful stuff specific to each project.

Thanks,

Rastus_Vernon added a comment.Via ConduitOct 28 2013, 5:01 AM

I agree with James Forrester's comment #0. Character input in particular languages should be possible, but through the universal language selector, and in all text inputs on the website.

I think the ideal way to handle en dashes and em dashes would be to make "--" be replaced by the wikitext parser by an en dash (–), "---" by an em dash (—) and "----" by an horizontal bar (―). The current horizontal rule syntax would become "-----" (it would require one more hyphen-minus because the current one would be taken by the horizontal bar character). There would therefore become part of the syntax. DokuWiki does this for en dashes and em dashes (https://www.dokuwiki.org/wiki:syntax#text_to_html_conversions), but not the others.

As I mentioned, this is what I think would be the ideal way to handle it. Practically, this is impossible because these sequences are already used in many pages for other purposes. An acceptable solution would, I believe, be to make the editor recognize those sequences and replace them automatically by the corresponding character. This would be reasonable in the VisualEditor and would be more efficient than putting a bar of characters under the edit box and hoping beginners will use it to insert dashes. However, since the sequence "--" is used in signatures on talk pages and possibly for other things in wikitext, it may be preferable to avoid doing these replacements in the source editor.

It might be a good idea to add sequences of characters for producing similar characters. As long as the sequences are well chosen, problems should not arise frequently, and this could reduce considerably the typographical burden put on beginners.

bzimport added a comment.Via ConduitOct 29 2013, 9:57 PM

eleriygarregwen wrote:

When I proposed a whole collection of sequences of characters to put into the ULS input method for Welsh, which uses a lot of diacritics, the comment from CYWP was that it would be a lot of work to memorise these character sequences, and it is not easy to find out about the ULS input methods.

The bar of characters under the edit box is very easy to understand, just by looking at it, and having a go. Almost no-one needs a help page on how to use it. Once you know what it does, there is nothing to memorise, and nothing to forget. Since simplicity is the aim with VisualEditor (to encourage new users), the bar of characters wins over anything more complicated, in my opinion.

He7d3r added a comment.Via ConduitOct 30 2013, 9:29 AM

If this selection tool is ever provided, please make sure users have an option not to load it at all (see bug 11130 for this problem in the context of MediaWiki:Edittools)

gerritbot added a comment.Via ConduitNov 2 2013, 4:26 PM

Change 90940 had a related patch set uploaded by Jforrester:
[WIP] Insert special character tool

https://gerrit.wikimedia.org/r/90940

Jdforrester-WMF added a comment.Via ConduitNov 2 2013, 4:54 PM

(In reply to comment #27)

If this selection tool is ever provided, please make sure users have an
option not to load it at all (see bug 11130 for this problem in the context
of MediaWiki:Edittools)

It's very unlikely that we'll start adding per-user preferences to switch things like this on and off (some VE version of bug 9754); that's a very complex and confusing thing to allow.

Note BTW that bug 11130 is complaining about assuming users have JS (already known inside VE), lack of ability to cache (fixed by VE's nature), and taking up lots of screen estate (fixed by how we'll do this); I don't think it's a direct comparison. I'd also note that bug 11130 hasn't been done despite it being filed 6 years ago, which strongly suggests that it's not a priority.

Juandev added a comment.Via ConduitNov 5 2013, 8:32 AM

Well, when creating Featured or Quality articles we (on cs.wp) looks to have them correct from typograpical point of wiev. The problem is that Czech keyboard doesnt include all characters needed, as it includes other characters, which are not native to English. The most common missing characters would be *, #, –,  . So if not all it would be nice to have in VE or nearby the table with some most used characters. E.g. list items are present in English keyboard, while – suprisingly not.

Amire80 added a comment.Via ConduitNov 5 2013, 2:50 PM

Just for reference: Either I didn't read all the comments well enough or nobody quite mentioned the fact that all modern desktop operating environments have a built-in a Character Map tool: Windows (with pretty full Unicode since NT 4), Gnome, KDE, and Mac OSX (called "Character viewer").

That said, I am very well aware of the fact that pretty much nobody but me actually uses it. A very surprising number of people, however, do use the character insertion tool in their word processors. The one in Microsoft Word is not very different from Windows' Character Map, which in my mind makes it redundant, but apparently I'm special.

Anyway, yes - a character insertion tool should be there.

But, two points:

  1. There's one already in Vector, and it's quite good. It can be used for research - which characters are used most frequently in which project. Adding statistics for this is quite easy now with EventLogging.
  1. It may be easier to insert at least some characters using the keyboard layout provided with ULS. It is disabled now in VE because of some bugs (can anybody help me with the numbers?), but these can be fixed, of course. If we know which characters are really needed (see point 1 above), we can very easily add them to the keyboard layouts.
Jdforrester-WMF added a comment.Via ConduitNov 5 2013, 5:58 PM

(In reply to comment #31)

Just for reference: Either I didn't read all the comments well enough or
nobody quite mentioned the fact that all modern desktop operating
environments have a built-in a Character Map tool: Windows (with pretty
full Unicode since NT 4), Gnome, KDE, and Mac OSX (called "Character
viewer").

I said this, but everyone said it wasn't sufficient for them. :-(

  1. It may be easier to insert at least some characters using the keyboard layout provided with ULS. It is disabled now in VE because of some bugs (can anybody help me with the numbers?), but these can be fixed, of course. If we know which characters are really needed (see point 1 above), we can very easily add them to the keyboard layouts.

Possibly. The main problem is that "ULS" is about twenty different tools, and some of them don't work in VisualEditor (yet). I doesn't seem possible to enable some of them and not others, however.

Juandev added a comment.Via ConduitNov 5 2013, 9:55 PM

(In reply to comment #32)

(In reply to comment #31)
> Just for reference: Either I didn't read all the comments well enough or
> nobody quite mentioned the fact that all modern desktop operating
> environments have a built-in a Character Map tool: Windows (with pretty
> full Unicode since NT 4), Gnome, KDE, and Mac OSX (called "Character
> viewer").

I said this, but everyone said it wasn't sufficient for them. :-(

Lets say it differenty. Visual Editor Team was made to remove obstacles in editing, to simplify the work of Wikipedia contributors. Even there are voices, that it is a wrong way and newbies should study wikitext WFM goes the way of its goal. To offer all human knowledge to give to everybody. Our aim here is to collect the knowledge of all people. Logically higher amount of obstacles reduces the number of Wikipedians. Our aim is not to push newbies to study wikitext, internal policies or typography. Our aim is to harvest their knoledge. Thus in the new trend we are removing obstacles. So if not as a default, what about as an option to define your own characters (this was always possible in wp)? Than we can tell newbies, look here we place – - its not present in VE default, but you can install an extension to your VE.

bzimport added a comment.Via ConduitNov 5 2013, 9:59 PM

dgerard wrote:

Juan - I think that would quite definitely require actual usability lab testing. As would many of the suggestions in this bug.

At present, however, there is no way to enter non-keyboard characters unless you're much more familiar with your operating system than almost any user is. So doing nothing isn't really an option either.

Juandev added a comment.Via ConduitNov 24 2013, 9:17 AM

(In reply to comment #34)

At present, however, there is no way to enter non-keyboard characters unless
you're much more familiar with your operating system

Well, how doest it come, there is no way? What about to load character box available for editing on some wikies in MonoBook?

At present, however, there is no way to enter non-keyboard characters unless
you're much more familiar with your operating system than almost any user is.

Do you have any statistic for this statment? I would say that most of the users are not familiar with that. But I should include it is my point of view.

TheDJ added a comment.Via ConduitNov 28 2013, 2:49 PM

Just for reference: Either I didn't read all the comments well enough or nobody
quite mentioned the fact that all modern desktop operating environments have a
built-in a Character Map tool: Windows (with pretty full Unicode since NT 4),
Gnome, KDE, and Mac OSX (called "Character viewer").

On Mountain Lion, it's even simpler. (Straight in the edit menu, ctrl-cmd-space).

I mentioned it, but the problem is indeed:
1: no one knows them/uses them
2: there is no trigger in HTML5/JS to open the panel

So some ideas:
1: Educate the user: Add a 'more characters' link in the insert symbol editor, that detects OS of user and links to a page explaining how to access this panel on his OS.
2: Lobby HTML5 WG to expose these panels to JS

bzimport added a comment.Via ConduitNov 28 2013, 3:01 PM

dgerard wrote:

"Educating" the user strikes me as a sign of failure for a supposedly "easier" UI. The "harder" UI has it right there, after all.

gerritbot added a comment.Via ConduitDec 6 2013, 6:55 AM

Change 90940 merged by jenkins-bot:
Insert special character tool

https://gerrit.wikimedia.org/r/90940

Juandev added a comment.Via ConduitDec 17 2013, 6:07 PM

(In reply to comment #36)

1: no one knows them/uses them

Well, that disscutalbe, but I tend to think the same. No one knows them, because you dont need to know them for normal sw use.

So some ideas:
1: Educate the user: Add a 'more characters' link in the insert symbol
editor,
that detects OS of user and links to a page explaining how to access this
panel
on his OS.

Why to make it more difficult for users. We want them to attract to Wikipedia and we wont do do the environment as easy as possible. Our goal is not to educate them further things.

Ironholds removed a subscriber: Ironholds.Via WebFeb 18 2015, 6:10 PM

Add Comment