Notes from today's story time:

• It's possible to use the dialog to add a template with subst:. In this case the only information we have is this prefix (plus the fact that a page with this name can't be found).
• There are also situations where the template name is auto-generated via other templates. In this case the template "name" might be gibberish including forbidden characters like {. This can't be created in VE, only edited. Questions:
  • How to detect these cases? Is looking for the characters {|} good enough? (Again, make sure the page doesn't exist before looking for anything else.)
  • Should the same message be used?

Example to play around with:

• Type subst:example when adding a template.
• In wikitext mode type {{ {{echo | example }} }}. Make sure both Template:Example and Template:Echo exist to make this work. Template:Echo contains nothing but {{{1|}}}. VisualEditor is able to resolve this situation and acts as expected.
• This is different when one of these templates doesn't exist any more, which can happen when editing e.g. old revisions. To reproduce this type something like {{ {{foo}} }} in wikitext mode, switch to VE and edit this template. VE will show {{foo}} as the template name.

While writing the comment above I realized it would be possible to detect the prefix subst: (and maybe safesubst:) and strip it when the code looks for TemplateData. The template will be found then and the dialog gets all the extra TemplateData information it needs. Nothing else would change. The output still contains the subst:.

To do:

• Check if there are existing tickets about this. Check if there are reasons mentioned to not do this.
  • T51904: Support "substituting" templates in transclusion dialog is an umbrella ticket for other issues related to subst:.
  • T86705: Make the TemplateData fetch and fallback link work when the template is being substituted is exactly what we talk about here.
  • T241586: VisualEditor should pre-parse subst: wikitext in autovalues while loading templateData is about subst: in values, not in the template name.
• Check if the 2 prefixes mentioned are the only one we need to hard-code. (Also: Upper/lowercase? Extra whitespace?)
  • Yes, these are the only 2 we care about. See https://phabricator.wikimedia.org/source/mediawiki/browse/master/includes/MagicWordFactory.php$188.
  • These magic words are hard-coded to be case-insensitive, so we could hard-code this as well. In case we end using the API we should use the case-sensitive flag it gives us.

Updated help text based on story time. If it needs to be further generalized because it will apply to many more cases than subst, let me know.

I thought there was still the open question though of it would be easier to simply show the template when a user uses subst, instead of the 'error' message. This is definitely an improvement, so if it was same or less effort it could make sense. But if it's actually a lot extra, sticking to the ticket as-is should be fine. How should we define this while you're investigating? Should it be added to the task as a nice to have? @Lena_WMDE what do you think?

@Andrew-WMDE, the most reliable way I found to include all possibly localized versions of subst: and safesubst: (including if it can be case-insensitive) is to use the API.

I suggest an approach where we implement this in steps, each based on the previous one:

1. Show the same error message in both cases, both when the given template name is not even a valid title (e.g. because it contains { and other wiki syntax), as well as when it contains :.
2. Hard-code the canonical strings subst: and safesubst: and only resolve these. Also hard-code this check to be case-insensitive (this is hard-coded in the parser as well, see above). This covers already >90% of the use-cases (because we know most wikis use only the canonical form of subst:, e.g. dewiki).
3. Use the API to include localized versions of subst:.

Note that : is a valid character in titles. Still I think a trivial check should work just fine: We know already when a template doesn't exist. In this case check if the name contains : and show the error message.

In T290140#7361365, @thiemowmde wrote:

we know most wikis use only the canonical form of subst:, e.g. dewiki

I wonder about this. {{ers:...}} resolves exactly the same as subst on dewiki, and once it's saved there's no record of which keyword is used, so how do we know that only the canonical name is used, just by looking at documentation?

Change 721797 had a related patch set uploaded (by Andrew-WMDE; author: Andrew-WMDE):

[mediawiki/extensions/VisualEditor@master] Improve messaging for when a template is not found

https://gerrit.wikimedia.org/r/721797

[…] how do we know that only the canonical name is used, just by looking at documentation?

We don't. But:

1. From my experience. Which might be anecdotal. But I feel it's more considering I was a technical Wikipedian since about 2004. ;-)
2. Considering how often "{subst:" vs. e.g. "{ers:" is mentioned.

And even if it's not 90% but e.g. 50%, we can still make life easier for 50% of the users by hard-coding subst:.

Change 721822 had a related patch set uploaded (by Andrew-WMDE; author: Andrew-WMDE):

[mediawiki/extensions/VisualEditor@master] Drop substitution prefixes before querying for TemplateData

https://gerrit.wikimedia.org/r/721822

Change 722266 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/VisualEditor@master] Add tests for MWTemplateSpecModel.getTitle()

https://gerrit.wikimedia.org/r/722266

Change 721822 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Drop substitution prefixes before querying for TemplateData

https://gerrit.wikimedia.org/r/721822

Change 721797 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Improve messaging for when a template is not found

https://gerrit.wikimedia.org/r/721797

Maybe I missed something, but is there a final decision yet on how specific the detection for modifiers should be when we show the message? Or is "just" looking for the colon : fine for this task?

Change 722266 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Add tests for MWTemplateSpecModel.getTitle()

https://gerrit.wikimedia.org/r/722266

is "just" looking for the colon : fine for this task?

I'm not 100% sure yet. Let's try to narrow this down incrementally.

The problem I see is that we can't know what certain characters in a template name mean. A : might be syntax or not. Even { is not necessarily syntax. Even attempting to parse the string would not give us a definitive answer because the string is not going to be used in isolation, but in the context of template syntax – not to mention the rest of the page.

I suggest to do something like this:

• Make sure the page doesn't exist, e.g. via ve.init.platform.linkCache.getCached(). Continue only if it's missing.
• Check if the title is valid, i.e. can be parsed via new mw.Title(). (Warning, this might even throw an exception. Make sure to properly cover it with tests.)
• When the title is invalid, we can assume it contains wiki syntax and show the appropriate error message.
• When the title is valid, it might still contain wiki syntax. I'm not 100% sure how to proceed here. One idea is to simply look for a : character anywhere in the string and show the same error message a above.
• One remaining issue with this simplified approach is that it will report e.g. User:Foo as being wiki syntax. We can fix this relatively easily. We can use the result from the new mw.Title() call above and check if it's in an unexpected namespace. If it is, we know the string started with a valid namespace prefix, and the presence of a : is most certainly because of that. In this case show the default "missing page" error message with a red link.

Note that the subst: prefix detection we are working on as well does not really make any of this obsolete subst:{{fo}} or subst:User:Foo are examples.

This is sounding like a lot of effort to show a slightly more specific message. If you've found a solution, then feel free to ignore this message but I also just thought of the option to combine the messages to just cover all the situations instead of needing to detect subst specifically.

It could look like, "This template doesn't exist. If you're using a modifier (such as subst) with a template you know exists, use the undocumented parameter input to add fields and values. Otherwise, you can use the back button to return to the search."

Hope it's not too late to share this idea! Let me know what you think.

Change 722583 had a related patch set uploaded (by WMDE-Fisch; author: WMDE-Fisch):

[mediawiki/extensions/VisualEditor@master] Use getTemplateDataQueryTitle to distinguish template title cases

https://gerrit.wikimedia.org/r/722583

Change 722583 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Use getTemplateDataQueryTitle to distinguish template title cases

https://gerrit.wikimedia.org/r/722583

To reproduce the following examples, you need to enter the wikitext in wikitext mode, then switch to visual mode without saving the page.

• {{existing template|x=y}}
• {{subst:existing template|x=y}}
  • Link to the existing template page. Message might still depend on if TemplateData exists or not. But the link should always be there, and be blue.
• {{missing template|x=y}}
• {{subst:missing template|x=y}}
  • Error message "this template doesn't exist" with a red link.
• {{User:Foo|x=y}}
• {{subst:User:Foo|x=y}}
  • Strip the subst:. Check if the remaining prefix is a known namespace prefix. If it is, this is just a regular page name and we need to do the same as above for existing/missing pages.
• {{foo:bar|x=y}}
• {{#foo:bar|x=y}}
  • Check if the prefix before the : is a known namespace. If it's not, show the error message about "syntax"/"modifiers".
• {{{{foo}}|x=y}}
• {{subst:{{foo}}|x=y}}
  • Message about "syntax"/"modifiers". This can simply be based on the fact that the title is invalid.

Change 722612 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/VisualEditor@master] Consolidate messages about missing/bad templates

https://gerrit.wikimedia.org/r/722612

Change 722624 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/VisualEditor@master] Fix existing templates without TemplateData not showing a link

https://gerrit.wikimedia.org/r/722624

@ECohen_WMDE Sorry if these questions are already answered somewhere else:

• Is it intentional that we don't provide a red link to the template page, in case it doesn't (yet) exist? This breaks some workflows. One (rare) situation I can think of is when I start using a template before I create it. This is a common pattern in software development. Another situation is when I want to go to the missing page to check it's history and logs. Maybe it was deleted for a reason?
• Currently there is nothing shown when a template exists but does not have any TemplateData information. Not even a link. I believe this is a mistake. It works in the old dialog. Fixed in https://gerrit.wikimedia.org/r/722624.

Is it intentional that we don't provide a red link to the template page, in case it doesn't (yet) exist? This breaks some workflows. One (rare) situation I can think of is when I start using a template before I create it. This is a common pattern in software development. Another situation is when I want to go to the missing page to check it's history and logs. Maybe it was deleted for a reason?

Do mean as part of the message in addition to the redlink shown in the undoc param input? Didn't realize that it could be so important to workflows that it should be more visible/duplicated but if you think it should be included in both places I'm not opposed to it. In that case the message should be "[[ $Link to template page | Template:$Template name ]] does not exist."

Currently there is nothing shown when a template exists but does not have any TemplateData information. Not even a link. I believe this is a mistake. It works in the old dialog.

I don't think this is true? Here is a screenshot with the link:

It could look like, "This template doesn't exist. If you're using a modifier (such as subst) with a template you know exists, use the undocumented parameter input to add fields and values. Otherwise, you can use the back button to return to the search."

Any thoughts on the joint message question I posed above? T290140#7365858

[…] in addition to the redlink shown in the undoc param input?

Oh sorry, I missed/confused that because it's collapsed by default. Let's leave it as it is then.

This is when the template exists, doesn't contain <templatedata>, and the parameters can't be auto-generated (either because there are no parameters, or because it's e.g. a Lua template). That's not a rare situation. A simple solution is to show the existing message »The "…" template doesn't yet have a description, but there might be some information on the template's page.« in addition to the gray box. Or to add the template name as a clickable link to the gray box.

In your example above the auto-generation found at least 1 parameter.

"This template doesn't exist. If you're using a modifier (such as subst) with a template you know exists, use the undocumented parameter input to add fields and values. Otherwise, you can use the back button to return to the search."

The part "this template doesn't exist" is more confusing than helpful, in my opinion. All we see might be a strange snippet of wikitext. But it might still resolve to a valid template name. We can't be sure.

And why highlight the back button?

What about this? "Template documentation and parameter names (if existing) are not available when editing templates utilizing nested syntax or modifiers, such as subst."

Change 722612 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Consolidate messages about missing/bad templates

https://gerrit.wikimedia.org/r/722612

Change 722624 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Fix existing templates without TemplateData not showing a link

https://gerrit.wikimedia.org/r/722624

Just my 2 cents:

This is when the template exists, doesn't contain <templatedata>, and the parameters can't be auto-generated (either because there are no parameters, or because it's e.g. a Lua template). That's not a rare situation. A simple solution is to show the existing message »The "…" template doesn't yet have a description, but there might be some information on the template's page.« in addition to the gray box. Or to add the template name as a clickable link to the gray box.

Seems very reasonable to me, I would go for that.

"This template doesn't exist. If you're using a modifier (such as subst) with a template you know exists, use the undocumented parameter input to add fields and values. Otherwise, you can use the back button to return to the search."

The part "this template doesn't exist" is more confusing than helpful, in my opinion. All we see might be a strange snippet of wikitext. But it might still resolve to a valid template name. We can't be sure.

What about this? "Template documentation and parameter names (if existing) are not available when editing templates utilizing nested syntax or modifiers, such as subst."

I'm also in favor for the suggested variant. Or at least a version that does not say doesn't exist.

Confirming final wording from sprint review:

"Template documentation and parameter names (if existing) are not available when editing templates utilizing nested syntax or modifiers, such as subst."

Everything else as demo'd looks good and this ticket is complete.

Change 724346 had a related patch set uploaded (by Thiemo Kreuz (WMDE); author: Thiemo Kreuz (WMDE)):

[mediawiki/extensions/VisualEditor@master] Refine wording of \"template name with subst/syntax\" message

https://gerrit.wikimedia.org/r/724346

Change 724346 merged by jenkins-bot:

[mediawiki/extensions/VisualEditor@master] Refine wording of \"template name with subst/syntax\" message

https://gerrit.wikimedia.org/r/724346