Page MenuHomePhabricator

Visual editor table editing conflicts with IME
Closed, DeclinedPublic8 Story Points

Description

Find a table. And single click a cell to edit.

If you start to input without IME, it will start to replace the content in that cell as normal

However, if you have IME activated (I tested Sogou IME in Chinese, Baidu IME in Japanese, same result), the first character disappears.

Very annoying to do editing on zh.wiki

100% reproduceable.

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJan 22 2017, 11:48 AM

A showcase GIF

Esanders added a subscriber: dchan.Jan 22 2017, 1:20 PM

Pretty sure this was reported before - but can't find it.

Either way - as we use keydown to change modes, I think it will be hard to not affect the IME. Maybe we could use compsitionstart, but I'm sure there are IME's that won't play nicely, right @dchan?

Not sure what's the technology behind Google Docs/Spreadsheet, seems to work fine with IME there

Pretty sure this was reported before - but can't find it.
Either way - as we use keydown to change modes, I think it will be hard to not affect the IME. Maybe we could use compsitionstart, but I'm sure there are IME's that won't play nicely, right @dchan?

Looks like T49436 for Sogou?

Pretty sure this was reported before - but can't find it.
Either way - as we use keydown to change modes, I think it will be hard to not affect the IME. Maybe we could use compsitionstart, but I'm sure there are IME's that won't play nicely, right @dchan?

Looks like T49436 for Sogou?

I saw that one, but it doesn't specify which part is glitching.

In my experience everything is fine until I start to edit tables

Pretty sure this was reported before - but can't find it.
Either way - as we use keydown to change modes, I think it will be hard to not affect the IME. Maybe we could use compsitionstart, but I'm sure there are IME's that won't play nicely, right @dchan?

Looks like T49436 for Sogou?

I saw that one, but it doesn't specify which part is glitching.
In my experience everything is fine until I start to edit tables

Therefore, every CJK IMEs are affected?

Pretty sure this was reported before - but can't find it.
Either way - as we use keydown to change modes, I think it will be hard to not affect the IME. Maybe we could use compsitionstart, but I'm sure there are IME's that won't play nicely, right @dchan?

Looks like T49436 for Sogou?

I saw that one, but it doesn't specify which part is glitching.
In my experience everything is fine until I start to edit tables

Therefore, every CJK IMEs are affected?

I only tested:

Sogou IME
Microsoft IME
Baidu Japanese IME
Microsoft Japanese IME

They're all broken. Can't speak for others.

dchan added a comment.Jan 23 2017, 6:00 AM

Thanks for the report! And thanks especially for the very helpful GIF. What browser are you using?

Unfortunately, the feature (typing into a table cell to open the selection) cannot be made to work properly on Firefox <51, because of a bug that causes a segfault: see T86589 and https://bugzilla.mozilla.org/show_bug.cgi?id=1230473 for further explanation.

Otherwise, it works on some browser+IME combinations, e.g. fcitx Pinyin on Chrome / Linux. But composition event sequences vary (hugely) between browser+IME combinations (and even different versions of the same browser+IME) , so it would not surprise me if you have uncovered another case we need to fix :-)

@Esanders, ve.ce.Surface#onDocumentCompositionStart already calls #handleInsertion (except on Firefox, because of the segfault described above), which changes table selections to linear selections to allow text input. It's possible this selection change closes the candidate window in the affected IMEs.

Sorry I didn't specify, I use Chrome 56.

dchan added a comment.Jan 23 2017, 6:32 AM

I can reproduce the behaviour on Chrome 55 on Windows 8 with Microsoft Japanese IME. On that IME+browser, the event sequence from typing 'n' then 'i' is as below. Note that 'compositionstart' is the first thing we see (apart from the fake keydown).

[
{"imeIdentifier":"","userAgent":"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36","startDom":""},
{"seq":0,"time":4.768,"action":"sendEvent","args":["keydown",{"keyCode":229}]},
{"seq":1,"time":4.776,"action":"sendEvent","args":["compositionstart",{}]},
{"seq":2,"time":4.78,"action":"changeText","args":["n"]},
{"seq":3,"time":4.78,"action":"changeSel","args":[0,1]},
{"seq":4,"time":4.78,"action":"sendEvent","args":["input",{}]},
{"seq":5,"time":4.816,"action":"changeSel","args":[1,1]},
{"seq":6,"time":4.816,"action":"endLoop","args":[]},
{"seq":7,"time":4.876,"action":"sendEvent","args":["keyup",{"keyCode":78}]},
{"seq":8,"time":4.888,"action":"endLoop","args":[]},
{"seq":9,"time":6.613,"action":"sendEvent","args":["keydown",{"keyCode":229}]},
{"seq":10,"time":6.613,"action":"changeText","args":["に"]},
{"seq":11,"time":6.613,"action":"sendEvent","args":["input",{}]},
{"seq":12,"time":6.661,"action":"endLoop","args":[]},
{"seq":13,"time":6.689,"action":"sendEvent","args":["keyup",{"keyCode":73}]},
{"seq":14,"time":6.705,"action":"endLoop","args":[]},
{"seq":15,"time":12.321,"action":"sendEvent","args":["input",{}]},
{"seq":16,"time":12.321,"action":"sendEvent","args":["compositionend",{}]},
{"seq":17,"time":12.497,"action":"changeSel","args":[5,5]},
{"seq":18,"time":12.497,"action":"endLoop","args":[]}
]

dchan added a comment.Jan 23 2017, 7:05 AM

Filibuster log of typing 'n' on Chrome 55 on Windows 8 with Microsoft Japanese IME

. Note how the wide 'n' disappears just before step 12426 (a second "input" event, even though only one key was pressed).

dchan added a comment.Jan 23 2017, 9:25 AM

From the filibuster log, I think it's likely we can't fix this :( It seems that when we change the selection in response to compositionStart, the immediate reaction of the IME in question is to close the candidate window (and hence effectively forget the first keystroke).

But perhaps we can mitigate this problem by changing the behaviour of Tab. Currently, it leaves the selection "outside" the next cell (i.e. a one-cell TableSelection), so the user of an affected IME has to enter *every* cell with a lost keystroke. We could change this so that Tab leaves the cursor "inside" the next cell (i.e. a LinearSelection). That way a user editing a number of adjacent cells would only encounter this problem once.

Change 336648 had a related patch set uploaded (by Esanders):
Stay in cell editing mode when pressing table inside a cell

https://gerrit.wikimedia.org/r/336648

Jdforrester-WMF triaged this task as High priority.
Jdforrester-WMF set the point value for this task to 8.

Change 336648 merged by jenkins-bot:
Stay in cell editing mode when pressing table inside a cell

https://gerrit.wikimedia.org/r/336648

dchan closed this task as Declined.Feb 10 2017, 12:45 AM

Change 336648 (which I've just merged) makes Tab step directly into the next cell. So you still get this problem, but only once when editing multiple adjacent cells.

I think that's probably the best we can do on this, until/unless Javascript/IME integration improves. (See https://w3c.github.io/input-events/ for current activity in this area).

Restricted Application removed a subscriber: Liuxinyu970226. · View Herald TranscriptFeb 10 2017, 12:45 AM
fireattack added a comment.EditedFeb 10 2017, 2:53 AM

But neither Microsoft's Office online (I tried Excel Online) or Google Docs (tried Spreadsheet) have this IME problem.

How did they solve it under current W3C standards?

dchan added a comment.Feb 10 2017, 8:53 AM

Their selections are much more faked than VisualEditor's - when inputting text, we have a real (native) cursor inside the ContentEditable block node where the text will live, whereas they don't.

It's not an inherently ridiculous idea for VisualEditor to move to using faked selections, and text insertion to a staging area outside the document structure being edited (indeed, VisualEditor originally used something similar several years ago). But it would be a big change with certain disadvantages (e.g. it's harder to integrate with browser spellchecking, mobile selection etc). A big decision, with this bug being only one comparatively small consideration. I accept it's annoying for the users affected, though :(

Oh Thanks for the detailed reply!

I was just curious about the tech behind the scenes, not trying to push.

Change 337417 had a related patch set uploaded (by Jforrester):
Update VE core submodule to master (2f38291c4)

https://gerrit.wikimedia.org/r/337417

Change 337417 merged by jenkins-bot:
Update VE core submodule to master (2f38291c4)

https://gerrit.wikimedia.org/r/337417

Liuxinyu970226 moved this task from Backlog to Closed on the Chinese-Sites board.Feb 18 2017, 3:25 AM