Page MenuHomePhabricator

TemplateData's custom format does not support the tab character
Closed, InvalidPublicBUG REPORT

Description

Steps to replicate the issue

Write in a template documentation page:

<templatedata>{
	"description": "Lorem ipsum",
	"format": "{{_\n	| _ = _\n}}",
	"params": {
		"foo": {
			"label": "Foo",
			"description": "Foo ipsum",
			"required": false
		}
	}
}</templatedata>

or

<templatedata>{
	"description": "Lorem ipsum",
	"format": "{{_\n\t| _ = _\n}}",
	"params": {
		"foo": {
			"label": "Foo",
			"description": "Foo ipsum",
			"required": false
		}
	}
}</templatedata>

What happens?:

Syntax error in JSON.

What should have happened instead?:

The table of template parameters should appear.

Event Timeline

Change #1309771 had a related patch set uploaded (by Anirudh_K; author: Anirudh_K):

[mediawiki/extensions/TemplateData@master] Fix custom format validation for tab characters

https://gerrit.wikimedia.org/r/1309771

Pushed a fix to Gerrit. Kindly Review
The first example fails because literal tabs are invalid JSON (they have to be escaped as \t).
The second example with \t failed because the regex in TemplateDataValidator only checked for spaces. I've updated it to allow [ \t]*

The lack of support may have been intentional, as it's difficult to type tabs in the wikitext editor (since the Tab key navigates the interactive elements of the page in the browser). This could lead to a disconnect between people who use the wikitext and the visual editor. I'm not sure if adding it is desirable.

@matmarex The format field only indicates a preferred format for VisualEditor; it does not restrict how users write the template invocation in wikitext. When editing wikitext, users can still choose whichever format is most convenient for them.

The same concern would apply to other formats that are already supported, for example:

{{_\n|_______________ = _\n}}\n

This format is arguably even more difficult to maintain manually, especially when editing from a smartphone, yet it is still a valid use case for the format field.

I don't think it's the same concern. The tab character is impossible to type without copy-pasting for most users. And while users can use a different format in wikitext than the specified one, visual editor edits are going to change the template to the specified format when the template is edited, leading to dirty diffs, which are a bad experience.

Thanks for raising this @matmarex. To be clear, this patch doesn't change how VisualEditor generates or normalizes the format field. It only fixes validation so that a \t a user has deliberately written in wikitext isn't rejected as invalid JSON. Right now that's a hard failure with a confusing error, regardless of whether we think tabs are a great choice for hand-editing.
If the concern is that Visual Editor might rewrite format to include tabs and create dirty diffs, I think that's a separate question Visual Editor's serialization behavior from whether the validator should accept valid, escaped JSON. Happy to loop in someone from the VE side if we need to confirm Visual Editor won't introduce tabs unprompted.
If the consensus is that tab support shouldn't be allowed at all (not just a bug in validation), I'm fine abandoning this and instead improving the error message so it's clear why the tab was rejected. Let me know which direction makes sense.

I'm afraid I don't even understand the problem. When I test the given example with the properly escaped \t I get the expected error message:

Property "format" is expected to be "inline", "block", or a valid format string.

A raw tab character in a JSON string value really is a syntax error. There is nothing we can do to "improve" this error message.

@thiemowmde

The issue is that it's currently possible to write a template like this:

{{Foobar
	| hello = world
	| wind = surf
}}

(where the indentation uses tab characters).

This means that a tab character is a valid space character in wikitext. If we wanted to forbid tab characters, we would have to reject markup like the example above; it makes little sense to introduce additional rules that apply only to the format string.

An extension like TemplateData cannot and probably shouldn't influence how wikitext behaves. Sure you can add tab characters anywhere in your wikitext when you really want to. But this doesn't mean that a VisualEditor feature needs to actively encourage this.

Instead of making assumptions about what is and what is not a bug, can we go back multiple steps and explain what the use case is? On which wiki do you have this problem? Which template? Can you provide example links? What is the benefit you are hoping for? How many users are affected?

An extension like TemplateData cannot and probably shouldn't influence how wikitext behaves.

Exactly: it should simply comply with wikitext's rules.

But this doesn't mean that a VisualEditor feature needs to actively encourage this.

Complying with how a language works does not mean encouraging or discouraging anything. Those are matters that are usually addressed in documentation (e.g. “Due to the difficulty of typing tab characters, and the difficulty of manually maintaining alignment, we discourage formats such as {{_\n|_______________ = _\n}}\n and {{_\n\t| _ = _\n}}.”). The rest is ultimately up to local communities, template authors, and common sense.

On which wiki do you have this problem? Which template? Can you provide example links? What is the benefit you are hoping for? How many users are affected?

At the moment, nowhere. I often use tab characters myself in vertically formatted templates, but I am happy for people to use whichever style they prefer.

Recently I was experimenting with the format field in order to take some decisions: The <templatedata>...</templatedata> JSON object is written in English and its syntax is not always easy to remember for people who do not use Visual Editor (I always forget it myself and end up copying and pasting it from other templates). Therefore on the Latin Wikipedia, I recently created a substituted template that allows writing a <templatedata>...</templatedata> tag using Latin parameter names and a simpler syntax. In short, you write

{{subst:De formula adhibenda

	| summarium = Use this template to indicate that an article is in need of cleanup.
	| dispositio = horizontalis

	| titulus 1 = Example parameter 1
	| descriptio 1 = Example parameter description 1
	| forma 1 = textualis

	| titulus 2 = Example parameter 2
	| descriptio 2 = Example parameter description 2
	| forma 2 = textualis
	| exemplum 2 = Hello world

	| nomen 3 = date
	| titulus 3 = Month and year
	| descriptio 3 = The month and year that the template was added
	| forma 3 = textualis
	| exordium 3 = {{subst:Mora substitutionis|CURRENTMONTHNAME}} {{subst:Mora substitutionis|CURRENTYEAR}}
	| exemplum 3 = January 2013
	| praesentia 3 = suasa

	| nomen 4 = reason
	| synonyma 4 = 3
	| titulus 4 = Reason
	| descriptio 4 = The reason the article is in need of cleanup
	| forma 4 = textualis

	| nomen 5 = talk
	| synonyma 5 = talksection, talkpart
	| titulus 5 = Talk page section
	| descriptio 5 = The section of the talk page containing relevant discussion
	| forma 5 = textualis

	| ordo = 1, 2, date, reason, talk

}}

and it expands to

<templatedata>{
	"description": "Use this template to indicate that an article is in need of cleanup.",
	"format": "inline",
	"paramOrder": [
		"1",
		"2",
		"date",
		"reason",
		"talk"
	],
	"params": {
		"1": {
			"description": "Example parameter description 1",
			"label": "Example parameter 1",
			"required": false,
			"type": "string"
		},
		"2": {
			"description": "Example parameter description 2",
			"example": "Hello world",
			"label": "Example parameter 2",
			"required": false,
			"type": "string"
		},
		"date": {
			"autovalue": "{{SUBST:CURRENTMONTHNAME}} {{SUBST:CURRENTYEAR}}",
			"description": "The month and year that the template was added",
			"example": "January 2013",
			"label": "Month and year",
			"required": false,
			"suggested": true,
			"type": "string"
		},
		"reason": {
			"aliases": [
				"3"
			],
			"description": "The reason the article is in need of cleanup",
			"label": "Reason",
			"required": false,
			"type": "string"
		},
		"talk": {
			"aliases": [
				"talksection",
				"talkpart"
			],
			"description": "The section of the talk page containing relevant discussion",
			"label": "Talk page section",
			"required": false,
			"type": "string"
		}
	}
}</templatedata>

(I apologize for the mixture of English and Latin)

At the moment, |dispositio= accepts verticalis (producing "block") and horizontalis (producing "inline"). I am now considering allowing the raw format string directly, but I also thought it might be useful to provide shortcut names for some common formats. That is how I ended up here. Nothing particularly exciting, really.

At the moment, nowhere.

Ok, thanks. I updated the task accordingly to reflect this.

Complying with how a language works […]

I'm afraid this is confusing something. The format syntax is not wikitext. It's a special syntax I originally invented as part of my Auto-Formatter user script. The responsible WMF product team found it useful and (partially) adapted it via T138492. The syntax might generate wikitext, but that doesn't mean it needs to support all features that are supported in wikitext.

Tabs are generally not useful in wikitext. All communities I worked with so far agreed on this. Tabs are impossible to type in most contexts. They are invisible and create confusion. They are not guaranteed to align the same way for all users. There is not even a guarantee that the various wikitext editors use a monospace font.

The rest is ultimately up to local communities […]

That's why I'm asking for which community this feature request was created?

The <templatedata>...</templatedata> JSON object is written in English and its syntax is not always easy to remember for people […]

Are you aware of the interactive template data editor that was created to solve exactly this problem?

The syntax might generate wikitext, but that doesn't mean it needs to support all features that are supported in wikitext.

I cannot really comment on this. I think it should, but it is not up to me to decide.

Are you aware of the interactive template data editor that was created to solve exactly this problem?

I am, but I am not really comfortable with anything that is not wikitext (and many editors with me). I am happy that it exists though.

Ok, thanks. I updated the task accordingly to reflect this.

And that means closing the task as “Closed, Invalid”?

Yes, since there is apparently no real-world use case. This is working as intended. Removing noise from the various Phabricator boards this appears on helps people to focus on more relevant tasks that have an impact on users.

Change #1309771 abandoned by Anirudh_K:

[mediawiki/extensions/TemplateData@master] Fix custom format validation for tab characters

Reason:

THE TASK IS CLOSED

https://gerrit.wikimedia.org/r/1309771

In T431913#12114217, @thiemowmde wrote:
Yes, since there is apparently no real-world use case. This is working as intended.

Well, I do want to add {{_\n\t| _ = _\n}} to {{subst:De formula adhibenda}}, and I am also considering adopting it as the preferred format for {{Opus}} on lawiki. This is only an initial proposal; I will be happy to use a different format if the community reaches a different consensus.

What is not clear to me is why you would want to make Visual Editor incompatible with wikisource editing. In other words, why do you think that editors who prefer tab indentation should avoid using Visual Editor? People who prefer tabs will continue to use them regardless.

I'm afraid this doesn't add anything to the discussion. Nobody is blocked from using VisualEditor. Tabs are useful in code when you have an IDE and a linter that take care of them for you. Nothing similar exists in any of the wikitext editors I'm aware of. Most people don't even know how to insert a tab character. They won't understand what's going on and do weird, unhelpful things like using spaces to visually align their new template parameter with the existing ones. You cannot force them to stick to a formatting they are technically unable to type, unless you force them to use VisualEditor.

Typing difficulty is largely a property of the editing tools rather than of the character itself. Editors already provide shortcuts for characters that would otherwise be inconvenient to enter, and those shortcuts strongly influence what people actually use. For example, typing d͡z is usually much harder than inserting a TAB character, but the source editor provides d͡z through “Special characters > IPA”. If tab indentation were considered desirable for a particular use case, making it easier to insert would be a tooling problem rather than a limitation of the format syntax.

More importantly, I am not arguing that tab indentation should become the general recommendation. My point is simply that there are cases—particularly with large, vertically formatted templates—where the benefit of stronger visual indentation may outweigh the relative difficulty of entering tabs. Whether that trade-off is worthwhile should, in my opinion, be left to the local community or template author, rather than being rejected by TemplateData's validation.