Page MenuHomePhabricator

Cope with IMEs that make uncancellable alterations to DOM structural nodes
Open, Needs TriagePublic

Description

An input actions such as a keystroke can change the DOM tree in two ways: either native (i.e. the browser decides how to modify the DOM tree) or emulated (i.e. our Javascript code listens for the action, cancels it, then modifies the DOM tree itself).

Our current paradigm is that native input actions should only change DOM inline content (i.e. content inside a ContentBranchNode such as a paragraph or heading). In detail:

  • We depend on it being impossible for native actions to change DOM structure (e.g. splitting paragraphs).
  • For events that would normally cause structural changes, like Enter keydown or certain Delete/Backspace keydowns, we cancel the native event with preventDefault and then implement the desired behaviour in Javascript.
  • Therefore, when polling the DOM tree for changes, we only have to check for inline content changed in the current ContentBranchNode.

However there are popular mobile IMEs that break that paradigm. IME events can be uncancellable (so preventDefault doesn't work), and this can happen when the selection crosses a ContentBranchNode boundary. Therefore we have to revisit that paradigm and work out how to adapt cope with the widened range of behaviours.

As always with IMEs, the solution will involve the following aspects:

  • Surveying popular IME+language+browser+platform combinations to learn what kinds of behaviour can happen
  • Doing a mixture of observing mutations (i.e. synchronizing DOM -> data model) and fixing them up (i.e. synchronizing data model -> DOM)
  • Timing DOM changes to minimize disruption due to premature IME commits
  • Working very carefully with event sequencing (see ve.EventSequencer for instance)
  • Using knowledge of typical editing behaviour to judge what constitutes a disregardable edge case

These are in no particular order, and in fact feed back on one another.

Event Timeline

JTannerWMF subscribed.

Hey @dchan I will move this into upcoming for now, but if you are currently working on it please feel free to move it to the current workboard.

JTannerWMF added a subscriber: matmarex.

@matmarex mentioned in task T218545 At least one specific case was fixed (T217223), others might not be. Additional approaches: use mutation observer to detect structural deletions; make each ve.ce.ContentBranchNode track its own ve.ce.TextState. How can we move forward with this task?

Removing task assignee due to inactivity, as this open task has been assigned for more than two years. See the email sent to the task assignee on February 06th 2022 (and T295729).

Please assign this task to yourself again if you still realistically [plan to] work on this task - it would be welcome.

If this task has been resolved in the meantime, or should not be worked on ("declined"), please update its task status via "Add Action… 🡒 Change Status".

Also see https://www.mediawiki.org/wiki/Bug_management/Assignee_cleanup for tips how to best manage your individual work in Phabricator.