Page MenuHomePhabricator

Change approach to content sanitisation when switching from source to visual
Closed, ResolvedPublic

Description

Our current technique for switching is the following:

Take the wikitext and add one level of indentation:

Foo
bar
==baz==
{|
|quux
|}

->

:Foo
:bar
:==baz==
:{|
:|quux
:|}

Convert this to HTML

<dl>
  <dd>Foo</dd>
  <dd>bar</dd>
  <dd>==baz==</dd>
  <!-- very broken table: -->
  <dd><dl><dd></dd></dl><table><tbody><tr><td>quux<dd>|}</dd></td></tr></tbody></table></dd>
</dl>

then unwrap/de-indent the list down to paragraphs:

<p>Foo</p>
<p>bar</p>
<p>==baz==</p>
<!-- broken table -->

Pros:

  • Correctly converts single linebreaks into separate paragraphs
  • Correctly preserves some unsupported syntax, e.g. headings becoming <p>==baz==</p>

Cons:

  • Mangles other syntax, resulting in broken roundtrips when you switch back (e.g. tables, templates)
  • No warning shown to the user that something bad is about to happen

The alternative approach we have discussed is:

  • do a context-less switch of the wikitext as-entered
    • It looks like we will need to keep the wrap/unwrap step to support single linebreaks.
  • before loading VE, check the HTML for unsupported features: tables, headings, templates, images
  • if such a feature is found, refuse to switch or show a warning

Cons:

  • We still need an approach for converting single linebreaks into separate paragraphs

Implementation details

  1. When someone writes a comment in the tool's source mode that contains a template or a table
  2. And then attempts to switch to the tool's visual mode, they should observe
  3. A dialog that meets the following conditions:
    • If the comment written in source mode contains a table, the user should see a dialog that says:
      • TITLE: Visual mode disabled / BODY: Sorry, switching to visual mode is disabled because **table** syntax was detected in the comment you have written. To switch modes, please delete this syntax. [Learn more](mw:Extension:DiscussionTools/Reply_tool_visual_mode_limitations)."
    • If the comment written in source mode contains a template, the user should see a dialog that says:
      • TITLE: Visual mode disabled / BODY: Sorry, switching to visual mode is disabled because **template** syntax was detected in the comment you have written. To switch modes, please delete this syntax. [Learn more](mw:Extension:DiscussionTools/Reply_tool_visual_mode_limitations)."

Open questions

  • 1. In what – if any – cases should people be blocked from switching from source to visual?

People will not be able to switch from source to visual if the comment they are writing contains a template or a table.

  • 2. In the cases where people are blocked from switching from source to visual, what should be communicated to them? How should this communication be presented?

People will see a dialog* that informs them they are not able to switch to the tool's visual mode, why they are not able to switch to the tool's visual mode and what they can do to get around this block.

  • 3. In what cases should people be warned when attempting to switch from source to visual?

There will be no "warn" case.

  • 4. In the cases where people are warned when attempting to switch from source to visual, what should be communicated to them? How this communication be presented?

Moot; see "3." above.

  • 5. How should single linebreaks be converted into separate paragraphs?

Comments of this type will be handled as they currently are: wrapped and unwrapped when switching modes. See: T256150#6256308

  • 6. @Esanders can you think of a way to present the exception message in a way that allows people to simultaneously read said "exception message" and the comment they've written? As this patch is currently implemented, the exception message obscures the comment the editor has written.

Done

  • "Open questions" are answered
  • The tool works as specified in the "Implementation details" section

Event Timeline

Change 607611 had a related patch set uploaded (by Esanders; owner: Esanders):
[mediawiki/extensions/DiscussionTools@master] WIP Don't allow switching when unsupported content used

https://gerrit.wikimedia.org/r/607611

@Esanders: this is helpful to be able to see/play around with, [i]

A couple of resulting questions for you:

  • 1. Can you say more to the scenario quoted below? Asked another way: what can I try with the current prototype [i] to experience the case that has not yet been designed for? Is it a reference to what's described in T240696?
    • "We still need an approach for converting single linebreaks into separate paragraphs"
  • 2. As the patch has currently been implemented, it looks like people would be blocked from switching from source to visual were they to write something like what's described in T256333 [ii], what led you take this more restrictive approach?
    • "Restrictive" = prevent people from switching vs. warning them about the implications of doing so?

...I've also added an "Open questions" section to the task description.


i. http://patchdemo.wmflabs.org/wikis/000454e89aa54a2b691e983472a6a8fc/w/index.php/Talk:Main_Page?dtvisual=1
ii.

Hello
{{quote|world}}

We still need an approach for converting single linebreaks into separate paragraphs

I think this is a hard limitation of the approach I initially outlined, accordingly I think we need to go with a hybrid solution where:

  • Switching is blocked if templates/tables/.. are found
  • but we still do the wrap/unwrap trick to handle linebreak conversion, but we are more confident it won't break because we have prevented problematic syntax.

This is what the patch you are testing does.

Is it a reference to what's described in T240696?

Yes.

  1. As the patch has currently been implemented, it looks like people would be blocked from switching from source to visual were they to write something like what's described in T256333 [ii], what led you take this more restrictive approach?

Per my comment T256333#6256303, I don't think we can confidently handle switching with templates, so it will be easier just to prevent it.

We still need an approach for converting single linebreaks into separate paragraphs

I think this is a hard limitation of the approach I initially outlined, accordingly I think we need to go with a hybrid solution where:

  • Switching is blocked if templates/tables/.. are found
  • but we still do the wrap/unwrap trick to handle linebreak conversion, but we are more confident it won't break because we have prevented problematic syntax.

This is what the patch you are testing does.

Roger that.

  1. As the patch has currently been implemented, it looks like people would be blocked from switching from source to visual were they to write something like what's described in T256333 [ii], what led you take this more restrictive approach?

Per my comment T256333#6256303, I don't think we can confidently handle switching with templates, so it will be easier just to prevent it.

Understood.

Open questions
Documenting the answers to the task description's "Open questions" that @Esanders, @iamjessklein and I talked about earlier today...

  • 1. In what – if any – cases should people be blocked from switching from source to visual?

People will not be able to switch from source to visual if the comment they are writing contains a template or a table.

Rationale:

  • We are assuming editors will value having certainty the content they write will be A) represented in the tool's text input and B) published to the page in the ways they expect more than they will value the tool being more "permissive" and accordingly, less reliable considering it is not possible for the tool to treat templates, whose behavior can vary widely, the same.*

*Note: new syntax for multi-line comments (T246960)will eliminate the need for this constraint. Said another way: new syntax for multi-line comments will enable people to include tables and multi-line templates within comments and switch between the tool's two input modes without the content you write being affected.

  • 2. In the cases where people are blocked from switching from source to visual, what should be communicated to them? How should this communication be presented?

People will see a dialog* that informs them they are not able to switch to the tool's visual mode, why they are not able to switch to the tool's visual mode and what they can do to get around this block.

Note: ideally, the tool would be able to instantly know when someone has typed a template or a table and adjust the interface accordingly (e.g. disable switching); however, b/c this "check" happens on the server, it is difficult to deliver this functionality.

  • 3. In what cases should people be warned when attempting to switch from source to visual?

There will be no "warn" case.

  • 4. In the cases where people are warned when attempting to switch from source to visual, what should be communicated to them? How this communication be presented?

Moot; see "3." above.

  • 5. How should single linebreaks be converted into separate paragraphs?

Comments of this type will be handled as they currently are: wrapped and unwrapped when switching modes. See: T256150#6256308

ppelberg updated the task description. (Show Details)

@Esanders I added the following to the task description:

  • "Implementation details" to
  • An additional question for you in the "Open questions" section (also quoted below)

  • 6. @Esanders can you think of a way to present the exception message in a way that allows people to simultaneously read said "exception message" and the comment they've written? As this patch is currently implemented, the exception message obscures the comment the editor has written.
  • 6. @Esanders can you think of a way to present the exception message in a way that allows people to simultaneously read said "exception message" and the comment they've written? As this patch is currently implemented, the exception message obscures the comment the editor has written.

Per the conversation we had during our team's 30-June standup, this message will continue to be presented to people as a modal.

Reason: the software doesn't know what people have written the tool's text input area until they attempt to save or switch modes. This means the software can't detect, in real-time, whether a drafted comment contains a template or a table.

Next steps:

  • Implement different modal messages for tables and templates.
  • Implement revised modal messages. See "Implementation details" in the task description.

To switch modes, please delete this syntax.

This sounds like we are encouraging the user to delete the syntax, which in the case of single line templates is not the case.

I think it is implied from the message that deleting the syntax is one possible option, but the other acceptable option is just to not switch.

  1. @Esanders can you think of a way to present the exception message in a way that allows people to simultaneously read said "exception message" and the comment they've written? As this patch is currently implemented, the exception message obscures the comment the editor has written.

As discussed offline, the modal dialog avoids two issues:

  • When would we hide a non-modal warning? We don't know if the issue has been resolved until they try to switch again.
  • Showing a non-modal warning could move controls on the page around.

These are not impossible challenges, but possible not worth the effort for this edge case warning.

The page at https://www.mediawiki.org/wiki/Extension:DiscussionTools/Reply_tool_visual_mode_limitations should exist before this is deployed.

Note there are now three cases we block which should be documented at that page:

  • Tables
  • Templates
  • Extensions, including references (see T251633)

Change 607611 merged by jenkins-bot:
[mediawiki/extensions/DiscussionTools@master] Don't allow switching when unsupported content used

https://gerrit.wikimedia.org/r/607611

matmarex removed a project: Patch-For-Review.
matmarex subscribed.
  1. @Esanders can you think of a way to present the exception message in a way that allows people to simultaneously read said "exception message" and the comment they've written? As this patch is currently implemented, the exception message obscures the comment the editor has written.

As discussed offline, the modal dialog avoids two issues:

  • When would we hide a non-modal warning? We don't know if the issue has been resolved until they try to switch again.
  • Showing a non-modal warning could move controls on the page around.

These are not impossible challenges, but possible not worth the effort for this edge case warning.

We could use the same interface as we use when errors occur when posting a reply.

image.png (641×2 px, 104 KB)

  • It would be hidden when the user attempts to switch (or post a reply), same as the current error handling.
  • It indeed moves the comment editing area, but for me that seems less disruptive than covering the entire page.

I think this would be better. I really hate modal dialogs :)

I've written a patch for this, I could push it for review if you think this would be a worthwhile change.

JTannerWMF subscribed.

In Board Refinement @ppelberg made the decision to stick with the modal.

I'm re-opening this task to tweak the dialog copy after realizing the dialog copy that's been implemented does not match the copy written in the task description's "Implementation details" section.

Notice the second sentence – To switch modes, please delete this syntax. – is missing from each of the cases noted in the table below:

Actual dialog copyExpected dialog copy
Screen Shot 2020-08-04 at 1.27.00 PM.png (422×1 px, 54 KB)
TITLE: Visual mode disabled / BODY: Sorry, switching to visual mode is disabled because **template** syntax was detected in the comment you have written. To switch modes, please delete this syntax. [Learn more](mw:Extension:DiscussionTools/Reply_tool_visual_mode_limitations).
screenshot.png (392×1 px, 65 KB)
TITLE: Visual mode disabled / BODY: Sorry, switching to visual mode is disabled because **template** syntax was detected in the comment you have written. To switch modes, please delete this syntax. [Learn more](mw:Extension:DiscussionTools/Reply_tool_visual_mode_limitations).

We are closing this because we rather keep the message shorter to increase the chances someone reads the dialogue themselves as oppose to making the message longer and risking the user not reading it at all.