In T153315#2945065, @Samwalton9 wrote:

is there any evidence that users who use the visual editor first and then switch to source editor actually expect their pastes to bring formatting across? If that's user expectation then fair enough, but I'm very skeptical. To me, plain text source editing provides an expectation that the software isn't going to second guess you and introduce text you didn't explicitly type or paste.

We would have to first establish whether (non-dev) editors who start in the visual mode and then switch to the wikitext mode actually believe that they are engaged in "plain text source editing". I would not be at all surprised to discover that many of them expected the two modes of the same software to work the same way: If I paste the italicized title of a book into the visual mode, then it should keep the italics in place, and if I paste the same italicized title of the same book into the wikitext mode, then it should still keep the italics in place. Same software, same content being pasted, same results (just displayed on my screen differently).

In T153315#3003479, @Whatamidoing-WMF wrote:

In T153315#2945065, @Samwalton9 wrote:

is there any evidence that users who use the visual editor first and then switch to source editor actually expect their pastes to bring formatting across?

Several comments in the Feedback page point in that direction: 1 2 3 4 5 6 - plus Samwalton9's and Dvorapa's comments above. This matches my own experience as well; the whole point of using a wikicode editor is having direct and full control over the text, and any magic from the editor breaks that expectation.

We would have to first establish whether (non-dev) editors who start in the visual mode and then switch to the wikitext mode actually believe that they are engaged in "plain text source editing".

Is that your primary use case, people accessing the 2017 editor by switching from the VE? It doesn't look like that in the beta, as the 2017 editor can be accessed directly from the "Edit source" link.

I would not be at all surprised to discover that many of them expected the two modes of the same software to work the same way: If I paste the italicized title of a book into the visual mode, then it should keep the italics in place, and if I paste the same italicized title of the same book into the wikitext mode, then it should still keep the italics in place. Same software, same content being pasted, same results (just displayed on my screen differently).

Nice hypothesis you have there. How are you going to test it? How will you test that it matches the expectations of users who enter the Wikitext editor directly? (Which I'd estimate is a majority of experienced Wikipedia editors, the same ones who currently use the classic editor)

I don't want that the work of power-users is disturbed by unnecassary magic in the technical source editor.

The simplest solution is to create a button for "paste formatted code". In the 2017 wikitext editor, other "magic" features for creating wikicode automatically (tables, sourcing and templates) are enabled by buttons in the toolbar. It would be inconsistent for the standard "copy/paste" tool (the one supported by the browser's standard key bindings) to behave differently, and introduce wikicode when pasting from web pages with rich format (but not when pasting from other sources, so its behavior would feel random).

And the users who started in the visual editor and switched to the wikicode editor have explicitly declared the intent of "I don't want to use the visual editor tool, I want to edit wikicode directly", so it should not surprise them that the new tool has its own specific set of behaviors - more so if the "paste rich content" button is highly visible in the toolbar in case that it is the behavior they want.

In T153315#3021511, @dialmove wrote:

The simplest solution is to create a button for "paste formatted code".

Browsers don't allow paste to trigger programmatically for security reasons. The user must initiate a paste using ctrl+v.

In T153315#3021583, @Esanders wrote:

Browsers don't allow paste to trigger programmatically for security reasons. The user must initiate a paste using ctrl+v.

Then let the user choose what will happen when they press ctrl+v. Don't force magic behavior on users who want the editor to work as a wikicode plain text area. The toolbar button then would mean "rich format pasting enabled / disabled".

Another use case to consider for you designers of this "feature", which just happened to me:

1. I want to create a piped link, so I type [[ |foo], where foo is the name of a disambiguation page.
2. I navigate to the the target article and copy its name, say, "bar", (of which "foo (special case)" is a redirect) from the bolded sentence of the lead section.
3. I paste the text from the clipboard into the blank space of my piped link, creating the totally useless broken link [['''bar'''|foo]], which I have to fix by hand.

Please don't let your answer be "you're doing it wrong, you should be using the insert wikilink feature", or I'll shoot myself.

In T153315#3028764, @dialmove wrote:

Please don't let your answer be "you're doing it wrong, you should be using the insert wikilink feature", or I'll shoot myself.

I don't think that I'd personally want to use the Link dialog for this (although it is convenient if you need to look up the name of the article). Your browser will let you paste plain text, with a key command of Command–Shift–V or Control–Shift–V. As the dev noted above, the "receiving" software doesn't have a lot of control over what your computer sends it.

(For myself, I usually copy from the URL rather than the first sentence of the article, because then my browser only has plain text. But then I get Really_Long_Names_With_Annoying_Underscores in my links.)

In T153315#3031416, @Whatamidoing-WMF wrote:

Your browser will let you paste plain text, with a key command of Command–Shift–V or Control–Shift–V. As the dev noted above, the "receiving" software doesn't have a lot of control over what your computer sends it.

Is there a reason for not making that the default behavior for Control+V? If the reason is "users expectations", how do you know what are the users expectations?

(For myself, I usually copy from the URL rather than the first sentence of the article, because then my browser only has plain text. But then I get Really_Long_Names_With_Annoying_Underscores in my links.)

Isn't it sad that the user needs to take such workarounds (I often temporarily paste rich text to a plain text file in Notepad, as an intermediate step to get plain text), when the tool could easily support doing the right thing as the default?

P.S. It looks like me and the other users posting at this thread are not the only one who prefer plain text as the default for copy/paste operations; it seems to be a frequent pain point, when it's got applications specifically built to solve it. [1] I've never met someone asking to paste formatted text when working with code. (Syntax highlighting doesn't count, it's a different beast entirely).

[1] Better Paste Takes the Annoyance out of Pasting Formatted Text

User experience

The following is a recent example of the user experience with the 2017 source editor that just happened to me when following normal Wikipedia administration procedures:

1. Open Wikipedia:Requested moves#Requesting technical moves to list a new request.
2. Following the process instructions, copy the template code listed in that section.
3. Move to the Uncontroversial technical requests subsection and enter the Edit source mode.
4. Paste the template code and fill in the fields
5. Press the Save button. Then press the Preview button.
6. Find out that the final page shows the template code, not the evaluated template.
7. Remember that you are supposed to press Ctrl+Shift+V every time you're working with anything other than plain text, or it will do random things.
8. Close the Preview view.
9. Delete the <nowiki> tags.
10. Preview again. Find out that the template now evaluates, but produces an error.
11. Scratch my head wondering what the hell may have happened this time.
12. Copy the template code again below the previous one, this time without any formatting magic, to compare the expected result with what the previous paste created.
13. Realize that the previous paste also had inserted italics code ('') around the identifiers for page names in the template. (Surely editors will expect format to be kept from the original page, right?)
14. Remove the italics code, and the second copy of the template.
15. Preview the page and find out that it finally does what it should have done three minutes earlier.
16. Save the page.
17. Uninstall the source editor in disgust.

I'll add another voice to the chorus saying that CTRL-V needs to be a plain text paste. The formatted paste behavior is disruptive.

In T153315#3004592, @dialmove wrote:

Nice hypothesis you have there. How are you going to test it? How will you test that it matches the expectations of users who enter the Wikitext editor directly? (Which I'd estimate is a majority of experienced Wikipedia editors, the same ones who currently use the classic editor)

That's the $64,000 question, isn't it? I'm not an expert, but it's obvious to even me that this kind of test could be heavily influenced by anchoring and similar effects. You and I have the old wikitext editors as our anchor (as did the editor who formatted the template syntax that you copied), but new editors don't.

So with my wikitext-based anchor, I'm thinking "it should work like what I'm used to" – which is that when I copy and paste formatted text, the result is always plain, and then I have to manually re-add all the formatting. But a new editor's anchor is the visual mode, and they're probably also thinking "it should work like what I'm used to". The difference is that "what I'm used to" for them is that when you copy and paste formatted text, the software magically figures out the formatting for you. For a first-time editor, whatever they see the first time is their initial anchor. So if they first paste something into the visual mode, then they'll expect "Wikipedia" to work that way. (New editors don't differentiate between tools. It's "the way you edit Wikipedia", not "the behavior of this specific tool, which is just one of more than a dozen ways to edit Wikipedia".)

Realistically, over time, I expect that most editors at most wikis will experience the visual mode first. Newly registered editors are already using the visual editor for between a third and half of mainspace edits at Wikipedias such as German, French, Spanish, Polish, Italian, Portuguese, Arabic, Turkish, and more, and the numbers are trending up over time. Right now, VisualEditor's built-in wikitext editor is being used primarily by old hands like us. I think, therefore, that the team is right not to rush into a decision how to address this. They're going to need information from all the editors, not just editors like you and me. The right answer may not be "always plain text, all the time". The right answer might be something complicated, like trying to figure out whether this is template syntax (and drop the formatting) or a pre-formatted bibliographic citation/book title/etc. (and keep the formatting).

In T153315#3112275, @Whatamidoing-WMF wrote:

In T153315#3004592, @dialmove wrote:

Nice hypothesis you have there. How are you going to test it? How will you test that it matches the expectations of users who enter the Wikitext editor directly? (Which I'd estimate is a majority of experienced Wikipedia editors, the same ones who currently use the classic editor)

That's the $64,000 question, isn't it? I'm not an expert, but it's obvious to even me that this kind of test could be heavily influenced by anchoring and similar effects. You and I have the old wikitext editors as our anchor (as did the editor who formatted the template syntax that you copied), but new editors don't.

Yes, you're right in that the new users might not have the same expectations. The problem is, there are still, literally, thousands of cases when the users are invited to paste wikitext. For instance, on some wikis, if they get blocked, they are informed they can appeal the block by pasting some wikitext and writing a reason. In the old wikitext editor and the visual editor, this works. However, in the new wikitext editor, it doesn't. So this is a difference from the default editor they're used to both for old and new editors.

Now, on the technical side, I have 2 questions:

• how can this be worked around right now? Would it work if we put the wikitext in <pre> or <code>?
• why it isn't possible to detect wikitext as, well, wikitext? If the VE can do it, can't that be ported to the current editor? That way, "real" HTML (say, a paste from a blog post) would keep its formatting, while wikitext would be detected and left alone.

Change 361731 had a related patch set uploaded (by Bartosz Dziewoński; owner: Bartosz Dziewoński):
[mediawiki/extensions/VisualEditor@master] In source mode, treat all pastes as plain text

https://gerrit.wikimedia.org/r/361731

Firstly I think we should be able to address the issue of pasting wikitext, as we do this already in visual mode. If you paste <b>{{foo}}</b> into visual mode you get an instance of the template, not some <nowiki>'d code. We should be able to re-use this code in source mode.

Secondly whatever we make the default action on paste (rich text or plain text) we should make the alternative more discoverable. A similar experience can be found in Excel/Calc where a modal dialog is presented asking how to format the just-pasted data. I think this would be too disruptive, but a non-modal offering to change the paste type after the event could work, e.g. "your text was just pasted with/without formatting, click here to remove/restore the formatting".

Change 361882 had a related patch set uploaded (by Esanders; owner: Esanders):
[VisualEditor/VisualEditor@master] Ignore covering annotations when looking for plain text pastes

https://gerrit.wikimedia.org/r/361882

Change 361882 merged by jenkins-bot:
[VisualEditor/VisualEditor@master] Ignore covering annotations when looking for plain text pastes

https://gerrit.wikimedia.org/r/361882

Change 361940 had a related patch set uploaded (by Jforrester; owner: Jforrester):
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (b528a5321)

https://gerrit.wikimedia.org/r/361940

Change 361890 had a related patch set uploaded (by Jforrester; owner: DLynch):
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (b528a5321)

https://gerrit.wikimedia.org/r/361890

Change 361940 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (b528a5321)

https://gerrit.wikimedia.org/r/361940

Change 361890 merged by jenkins-bot:
[mediawiki/extensions/VisualEditor@master] Update VE core submodule to master (b528a5321)

https://gerrit.wikimedia.org/r/361890

Change 361731 abandoned by Bartosz Dziewoński:
In source mode, treat all pastes as plain text

https://gerrit.wikimedia.org/r/361731

https://www.mediawiki.org/wiki/Topic:U0bhx94xejzyreee suggests that this bug may need to be re-opened.

In T153315#3698514, @Whatamidoing-WMF wrote:

https://www.mediawiki.org/wiki/Topic:U0bhx94xejzyreee suggests that this bug may need to be re-opened.

It was my comment. My problem is that if I copy the following text from here

<tt>{{tl|szszt}}<br /> Jó szócikk státuszt megkapta</tt>

I still get this:

<code><code><nowiki>{{</nowiki>[[Template:Szszt|szszt]]<nowiki>}}</nowiki></code></code>
<code>Jó szócikk státuszt megkapta</code>

In T153315#3698711, @Bencemac wrote:

My problem is that if I copy the following text from here

<tt>{{tl|szszt}}<br /> Jó szócikk státuszt megkapta</tt>

I can't find that snippet anywhere on the page or in the wikitext. Are you sure you got the details right?

Well, I was able to reproduce this another way, using the instructions in the task description.

1. Go to https://en.wikipedia.org/wiki/Template:Other_uses
2. Copy {{Other uses|Target (disambiguation)}} from the page whilst in read mode
3. Go to a random article and edit it using the 2017 editor
4. Paste the snippet you copied

Expected: You get {{Other uses|Target (disambiguation)}}

Actual: You get '''<code><nowiki>{{</nowiki>[null Other uses]<nowiki>|Target (disambiguation)}}</nowiki></code>'''

The problem is trying to balance the "I want to paste wikitext in literally as it is" with other use cases. Right now if I'm reading an article, and I copy, say, a link, then pasting it into the editor will get me the equivalent wikitext. That's really nice functionality that it would be sad to lose.

What I don't understand here is why it seemed to be working really nicely at balancing these a while ago but isn't any more.

I've done some more testing and it seems there is a difference between pasting text and html, which makes sense. Perhaps we're approaching this use case by the wrong end: would it be easier to detect when the user is copying wikitext and keep only the text, without the HTML "garbage"? This way we would help at least the js-enabled users (which are probably the vast majority anyway).

In T153315#3699098, @Strainu wrote:

I've done some more testing and it seems there is a difference between pasting text and html, which makes sense. Perhaps we're approaching this use case by the wrong end: would it be easier to detect when the user is copying wikitext and keep only the text, without the HTML "garbage"? This way we would help at least the js-enabled users (which are probably the vast majority anyway).

How would you detect that a user is copying wikitext? There are heuristics that you could come up with, but such a heuristic would frequently fail, be incredibly fragile, and hard to maintain properly.

The comment @Esanders made in T153315#3387085 about how it should work is clearly not what's happening here any more. We should wait for him to take a look before imagining solutions which may be worse or more complex in a lot of aspects.

The text in that example is actually complexly annotated, because 'Other uses' is a self-link.

In T153315#3699104, @Deskana wrote:

How would you detect that a user is copying wikitext? There are heuristics that you could come up with, but such a heuristic would frequently fail, be incredibly fragile, and hard to maintain properly.

The same way you detect wikitext when pasting/writing? I haven't looked at the code though, so I might be talking nonsense.

I still think that the new wikitext editor should not try to handle formatted pastes at all. At least, "paste as plain text" should be the default, like in every code editor in existence ever.

In my opinion, it's more important that you're able to paste wikitext than literaly anything else you could come up with.

I was trying to copy wikitext from the source editor to the source editor and got it full of nowiki tags – completely unexpected behavior. Someone asked a work around earlier: Ctrl+Shift+V worked for me.

Thanks for the tip with Ctrl+Shift+V. I was tearing my hair when I tried to manually archive a discussion section by copying it to an archive page. I agree with @dialmove and others above: This is not intuitive and should be fixed (e.g. by asking the user after pressing Ctrl+V whether he wants to convert the text or not).

In T153315#2945065, @Samwalton9 wrote:

I'm not sure if there's somewhere better to discuss this on Phab, but I agree with matmarex regarding copy/paste. It's very unintuitive to be working in a plaintext source editor and have the software be guessing at what you copied to add extra formatting you didn't ask for. I think the new wikitext editor is great but this feature made me go back to the old editor; I got into the habit of using Ctrl+Shift+V which ended up being more bother than the new editor was worth.

@Jdforrester-WMF is there any evidence that users who use the visual editor first and then switch to source editor actually expect their pastes to bring formatting across? If that's user expectation then fair enough, but I'm very skeptical. To me, plain text source editing provides an expectation that the software isn't going to second guess you and introduce text you didn't explicitly type or paste.

I agree; usually when I paste something into the wikitext editor I am pasting wikitext. If I want to paste formatted text, I paste it into the visual editor instead. I've been switching to the wikitext editor specifically to paste something and then going "oh yeah, this does this."

This is easily my top annoyance using wikieditor. I haven't turned off wikieditor but I've been toggling JS off to get to the old editor, saving, and then reopening. CTRL+SHIFT+V seems like it would be a better choice though so I'll try to remember that from now on :).

Has there been a regression in copy/pasting plain text? I found a sea of <nowiki> tags which did not happen earlier when I pasted a wall of wikitext into a new page on the wikitext editor. Like @Mvolz mentioned, I specifically switch to this mode for this purpose.

A few points here:

Firstly, please provide diffs for the offending edits that have nowiki tags in them, as I can't know what's going wrong without them. There are a bunch of times that people think there shouldn't be nowiki tags in their edits, but actually what they've pasted can't be accurately represented in wikitext without using nowiki tags. I want to know what's going wrong so that I know what needs to be investigated.

Secondly, please provide your browser and operating system. A lot of copy-paste bugs are highly specific to certain browser and operating system combinations, and I will probably be unable your issues without this information.

Thirdly, this task is about the 2017 wikitext editor available as a beta feature, so please be sure that that is what you're using, and not any of the older editor wikitext editors.

Thanks!

@Deskana: Just tested it again with Chrome 50 in Windows 7. (I know it's outdated, but I tested it with Chrome 64 earlier - same results) Both Wikitext 2017 (Beta) and Syntax-Highlighting (Beta) are enabled in my settings.

Opened this page twice, in two tabs:
https://de.wikipedia.org/w/index.php?title=Benutzer:Tkarcher/Spielwiese&veaction=editsource

1. Ctrl+A, Ctrl+C in one tab.
2. Ctrl+V in the same tab works as expected.
3. Ctrl+V in the second tab leads to wrongly escaped wikitext (see diff at https://de.wikipedia.org/w/index.php?title=Benutzer:Tkarcher/Spielwiese&diff=174517703&oldid=173969160)

@Tkarcher, could you please re-test this with Syntax Highlighting disabled? The bug might be in CodeMirror...

I did some more tests:

1. The bug as described above is also reproducable in Chrome 64 with Windows 10
2. @Whatamidoing-WMF: Same wrong behaviour with Syntax-Highlighting switched off. (no difference)
3. However, it's *not* reproducable with Firefox 58 or Edge 41.

So it's definitely browser dependent, and apparently not caused by CodeMirror.

Oh! The original bug reported by the task author (Copy {{Other uses|Target (disambiguation)}} from HTML, e.g. https://en.wikipedia.org/wiki/Template:Other_uses The paste logic detects this as non-plain text HTML (which it is) and so converts it to wikitext, so the {{ are <nowiki>'d) is also reproducable in Edge and Firefox. So the bug seems to be more prevalent in Chrome, but some cases can be reproduced in other browsers as well.

Interesting: When I copy

Paris}}

from the page mentioned above, pasting works fine. But when I copy

{{other uses|Paris}}

all my browsers will add <nowiki>s to it. So apparently the bold part triggers the wrong format detection (2017 Wikitext editing switched on, Syntax Highlighting switched off)

In T153315#4014300, @Deskana wrote:

Firstly, please provide diffs for the offending edits that have nowiki tags in them, as I can't know what's going wrong without them.

Nope, that won't help. We've known exactly what was wrong, and exactly how to fix it, since 2016. It's trivially easy to fix.

This bug is strictly a symptom of shoving the 2017Editor inside of VE. The 2017Editor is trying to do Visual pastes in Wikitext mode. Every editor who looks at this is saying the same thing. That is behavior is broken.

There are a bunch of times that people think there shouldn't be nowiki tags in their edits, but actually what they've pasted can't be accurately represented in wikitext without using nowiki tags.

That's exactly what everyone is complaining about. Stop doing that on CTRL-V pastes. The content gets mangled when you try to "represent" it in wikitext.

Just swap the behavior of CTRL-V and SHIFT-CTRL-V. Problem solved.

P.S. Now Flow's paste is broken too, because the wikitext mode was converted to use the 2017Editor.

@Deskana The example listed in the task is an appropriate example for me :).

In T153315#4014300, @Deskana wrote:

A few points here:

Firstly, please provide diffs for the offending edits that have nowiki tags in them, as I can't know what's going wrong without them. There are a bunch of times that people think there shouldn't be nowiki tags in their edits, but actually what they've pasted can't be accurately represented in wikitext without using nowiki tags. I want to know what's going wrong so that I know what needs to be investigated.

Secondly, please provide your browser and operating system. A lot of copy-paste bugs are highly specific to certain browser and operating system combinations, and I will probably be unable your issues without this information.

Thirdly, this task is about the 2017 wikitext editor available as a beta feature, so please be sure that that is what you're using, and not any of the older editor wikitext editors.

Thanks!

OK, here's one that I just did. I saved it so you can see the diff because luckily it was just in a sandbox anyway: https://en.wikipedia.org/w/index.php?title=User:Mvolz/Sennertia&type=revision&diff=828604981&oldid=828603637&diffmode=source

I'm on Chrome here.

1. Copied taxobox using Ctrl+C from https://en.wikipedia.org/wiki/Chaetodactylus?veaction=editsource from in the wiktext editor
2. Pasted the copied taxobox into the wikitext editor on User:Mvolz/Sennertia
3. It's no wiki-ed! Blast!

Then I used ctrl+shift+v instead :).

But now, super weird, I've tried replicating this doing the exact same thing and I cannot. Unfortunately I didn't have the console open at the time. This is definitely a bug. Next time this happens to me I'll try to do a more useful investigation.

Change 361731 restored by Bartosz Dziewoński:
In source mode, treat all pastes as plain text

Reason:
I remain convinced that this is the only correct solution to this task.

https://gerrit.wikimedia.org/r/361731

@Mvolz I can reproduce reliably when I copy it together with the newline afterwards (but not when I copy it with two newlines).

My patch above actually doesn't fix this case.

It seems like there are 2 related issues in this bug. One is the Wikitext editor converting formatting into the Wikitext (which might be helpful, if unexpected). The other is putting <nowiki> tags around any Wikitext (which really doesn't make any sense to me). I copy and paste Wikitext into Wikipedia articles all the time. The <nowiki> behavior is basically a blocker for me using the new Wikitext editor.

Change 361731 abandoned by Bartosz Dziewoński:
In source mode, treat all pastes as plain text

https://gerrit.wikimedia.org/r/361731