@Jdforrester-WMF Maybe the author wanted to say, that some functionality of Citoid currently existing in VE is missing in NWE

In T164954#3253275, @Dvorapa wrote:

@Jdforrester-WMF Maybe the author wanted to say, that some functionality of Citoid currently existing in VE is missing in NWE

Exactly. I wanted to say that the "re-use" is missing in NWE (I reported it here). I think it is a very useful function which would be great if you could enable in NWE. Is it possible?

I believe that James said above that this is "not possible", even though it would obviously be desirable. The Re-use tool in visual mode knows what all the existing refs are (existing names [if any] and entire contents), lets you search them, and, if you want to re-use one that is not already named, it will name the old one for you, in addition to inserting the <ref name="Example" /> code where you want to re-use it.

The "knowing" and "searching" and "naming" parts are not possible in wikitext mode.

However, perhaps a semi-manual "inserting" part could be created. It could insert <ref name="CHANGE THIS" />, similar to how the CharInsert tool can insert static, editable text in the older wikitext editors. You would still have to change the name of the ref, and name the older one, if it isn't already named.

@Deskana T174585: Reconsider offering an "edit" button for automatic citations, in NWE and this task are not apparently related. Why did you merge them?

In T164954#3724543, @Izno wrote:

@Deskana T174585: Reconsider offering an "edit" button for automatic citations, in NWE and this task are not apparently related. Why did you merge them?

They have the same solution. Adding the ability to manipulate citations in the 2017 wikitext editor will automatically unlock a bunch of different functionality, such as the ability to reuse and edit citations.

In T164954#3724616, @Deskana wrote:

In T164954#3724543, @Izno wrote:

@Deskana T174585: Reconsider offering an "edit" button for automatic citations, in NWE and this task are not apparently related. Why did you merge them?

They have the same solution. Adding the ability to manipulate citations in the 2017 wikitext editor will automatically unlock a bunch of different functionality, such as the ability to reuse and edit citations.

Maybe the same final solution? What was originally requested was to get us into the citation editor from the "are you sure this is the citation you want to insert?" screen.

Not a full-blown "I see any reference in wikitext and want to manipulate the parameters of that citation with a GUI rather than hand-typing it".

The latter solves the "I want to edit any template in wikitext using a GUI instead", as it happens.... While the former is already built from what I can tell (due to VE-implementation) and just needs the wiring reconfigured.

Impossible is a big word. It's definitely not easy, and its highly sensitive, because transclusion makes everything hard, but....

We already have the raw wiki text, we already have the names and params of all citation templates, some regex matching and done. You could even do parse api requests to get a preview. The ProveIt gadget shows that this can be pretty accurate, maybe not 100% complete, but pretty damn good.

See also: https://en.wikipedia.org/w/index.php?title=Super_Mario_Galaxy&action=edit&withJS=MediaWiki%3AGadget-ProveIt.js&withCSS=MediaWiki%3AGadget-ProveIt.css

Hard yes, different from VE yes, impossible... meh.

In T164954#3726728, @TheDJ wrote:

Impossible is a big word. It's definitely not easy, and its highly sensitive, because transclusion makes everything hard, but....

Happy to re-phrase to "impossible whilst meeting the fundamental performance requirements for people who don't have a high-end machine" if you prefer.

In T164954#3761519, @Jdforrester-WMF wrote:

Impossible is a big word. It's definitely not easy, and its highly sensitive, because transclusion makes everything hard, but....

Happy to re-phrase to "impossible whilst meeting the fundamental performance requirements for people who don't have a high-end machine" if you prefer.

It can all be done async and only when requested by the user though. So the fact that it's slow is much less problematic then. I mean, previewing a map or a graph is also very slow, but we don't need to do it all the time, making it acceptable to do it if that particular editor tool is open, right ?

In T164954#3761981, @TheDJ wrote:

In T164954#3761519, @Jdforrester-WMF wrote:

Impossible is a big word. It's definitely not easy, and its highly sensitive, because transclusion makes everything hard, but....

Happy to re-phrase to "impossible whilst meeting the fundamental performance requirements for people who don't have a high-end machine" if you prefer.

It can all be done async and only when requested by the user though. So the fact that it's slow is much less problematic then. I mean, previewing a map or a graph is also very slow, but we don't need to do it all the time, making it acceptable to do it if that particular editor tool is open, right ?

I don't understand. This would be a button that says "re-use a citation" that is always enabled, and when you click it freezes up for a few seconds as the article is round-tripped through the server for parsing, and then sometimes tells you that there's nothing to re-use after a half second or so (violating one of our principle design tenets), and the rest of the time lists the ones it could spot in the content? That sounds like a really terrible user experience.

If that were true, than ProveIt would be equally totally unusable.

How is this harder then syntax highlighting which we already have? For that you need to detect when <ref> is a ref tag (as opposed to, say, part of a comment or a nowiki block), and that's all you really need to find and edit references.

In T164954#3771980, @Tgr wrote:

How is this harder then syntax highlighting which we already have? For that you need to detect when <ref> is a ref tag (as opposed to, say, part of a comment or a nowiki block), and that's all you really need to find and edit references.

There's a lot of metadata that's required to offer full editing of templates and references. This metadata is used extensively in the visual editor, but it's not loaded when editing using the 2017 wikitext editor (or other wikitext editors) because it's not useful for anything.

So, if the user is editing wikitext, you could load this metadata speculatively, assuming the user may use it while editing at some point. If they don't use it, which is true in most cases, their bandwidth usage is higher, and the editor is less performant because it's loading data it doesn't need. Or, you could load the data on demand when the user tries to edit a reference, but the editor will have to wait for the response, which at best worsens the user experience slightly due to the small loading delays, and at worst on bad connections causes significantly delays.

As an aside, these issues are very serious if you look at them strategically. The disadvantages listed above are exacerbated if you're on a slow or lossy connection, or on an old or slow machine, which is potentially very bad if we're trying to attract new editors from emerging communities.

So, yes, @TheDJ is right that this is perfectly possible, but @Jdforrester-WMF is also right that this is complex and has the potential to cause a large number of issues.

We are not talking about full citation editing though, just reusing citations (basically parsing out the name="..." part of an existing ref tag and then adding <ref name="..." /> elsewhere).

The VE reuse dialog shows how citations render and you'd need a Parsoid roundtrip for that, but it seems reasonable that someone using the text mode would rather see the source code of citations, and there is no need for any backend request to get that.

In T164954#3775684, @Tgr wrote:

We are not talking about full citation editing though, just reusing citations (basically parsing out the name="..." part of an existing ref tag and then adding <ref name="..." /> elsewhere).

The VE reuse dialog shows how citations render and you'd need a Parsoid roundtrip for that, but it seems reasonable that someone using the text mode would rather see the source code of citations, and there is no need for any backend request to get that.

It wouldn't be quite that simple. If there's no name for the reference, which is the case for the majority of references, one would need to be added. We'd have to be careful not to add a reference name that's already in use on the page, which involves awareness of the entire wikitext content of the page and modifying it based on that. So, at that point we're talking about nontrivial client-side parsing of the wikitext of a page, which is incredibly fragile, and duplicative of Parsoid. That seems like a very bad road to go down to me.

In T164954#3777844, @Deskana wrote:

It wouldn't be quite that simple. If there's no name for the reference, which is the case for the majority of references, one would need to be added. We'd have to be careful not to add a reference name that's already in use on the page, which involves awareness of the entire wikitext content of the page and modifying it based on that. So, at that point we're talking about nontrivial client-side parsing of the wikitext of a page, which is incredibly fragile, and duplicative of Parsoid. That seems like a very bad road to go down to me.

I still don't get it. Per that argument, we can't do section editing in NWE either, yet we can. Perfection is the enemy of good here in my view.

In T164954#3777844, @Deskana wrote:

If there's no name for the reference, which is the case for the majority of references, one would need to be added. We'd have to be careful not to add a reference name that's already in use on the page, which involves awareness of the entire wikitext content of the page and modifying it based on that.

My point is that syntax highlighting already involves awareness of the entire wikitext content (to the extent required here, ie. being able to identify <ref> tags). Or do you mean that it's OK for syntax highlighting to be fragile and get edge cases wrong, while the same would be not acceptable when generating references?

Maybe this should be solved before T84936: Release VisualEditor-MediaWiki as "1.0"?