Page MenuHomePhabricator

Visual Editor removing whitespace from infoboxes by default
Open, Needs TriagePublic0 Estimated Story Points

Description

Problem statement

When you edit a template in the visual editor, it removes the old copy of the entire template and replaces it with a new copy of the whole template (even if you only edit one parameter). The visual editor always inserts templates according to the format specified in that template's documentation (look for a ==Template Data== section), or, if none exists, to the default formatting for that wiki.

Problem: Sometimes, the format specified in the template's documentation is not the format used in a specific article, or the format preferred by an individual editor. This can result in whitespace changes that make it difficult to read the diff.

Workaround

  • Correct the TemplateData for that template.
  • Make sure that the TemplateData and any sample copies of the template in the documentation are using the same format.

Explanation

Typing individual parameters on separate lines can mislead non-technical editors into thinking that each line is "one thing". As a result, people have an expectation that if they change "one thing" inside a template, then only "one thing" will be changed in the wikitext.

However, the entire template is "one thing", and changing any part of it causes all of it to be removed and replaced. This has some advantages (e.g., standardization, predictable ordering of parameters), but it can cause annoying diffs, especially if (a) it's early in the community's adoption of TemplateData, so most articles aren't already using the locally defined standard formatting, (b) the human-readable documentation doesn't match the TemplateData, so editors coping the wikitext samples are using a style that will be standardized later by the visual editor, or (c) someone changes the TemplateData to a different style (eventually triggering another round of whitespace changes, once per article).

Description from 2017

Hey all, so it seems recently Visual Editor has taken to removing whitespace from infoboxes by default. Ex: https://en.wikipedia.org/w/index.php?title=Ye_Hai_Mohabbatein&diff=806562609&oldid=805739169, https://en.wikipedia.org/w/index.php?title=Ishqbaaaz&diff=prev&oldid=806560482, https://en.wikipedia.org/w/index.php?title=2.0_(film)&diff=806958999&oldid=806721661, https://en.wikipedia.org/w/index.php?title=Bareilly_Ki_Barfi&diff=807726878&oldid=807726502. This is not constructive, as many editors and communities prefer that there be some element of infobox formatting to promote data legibility. For example, see https://en.wikipedia.org/wiki/Template:Infobox_film. Any help would be appreciated, because each of these edits is basically undoing a lot of editorss efforts with no discussion or policy/guideline-backed authority. Thanks,

Related Objects

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

The formatting for 'Infobox television' is 'unspecified', which I assume means leave whitespace alone, but the Parsoid team will need to answer this.

There is currently no an automated setting for space-aligned parameters.

I've been corrected below, you can use the following format:
{{_\n|_______________ = _\n}}\n

(with as many underscores as spaces are required)

Parsoid (which the visual editor uses) now normalises templates to the format specified by users in the template's template data. This was announced in Tech News. This change also added increased flexibility for users to define custom formats for templates, to specify whether templates should be on their own line or not, and other useful changes.

Both infobox television and infobox film have had "format": "block" in their template data for quite some time. If you do not wish for these infoboxes to have block formatting, I suggest removing the definition for them to have block formatting in their template data. There are now custom formats, such as aligning all parameter names to have a given length, that you can apply instead if you wish.

NB many users of Infobox television use a fixed width equal to the longest parameter in use, such that if a longer parameter is added, all the existing parameters need to be re-indented. TemplateData does not allow you to specify such a "variable-fixed width". Many also use simple block format with singles spaces, so whichever format is chosen, there is going to be some temporary disruption as usages are normalised.

The formatting for 'Infobox television' is 'unspecified', which I assume means leave whitespace alone, but the Parsoid team will need to answer this.

https://en.wikipedia.org/w/api.php?action=templatedata&titles=Template:Infobox_television&format=json returns "format":"block".

If it returns unspecified, I verified that Parsoid preserves the spaces. So, either the templatedata entry has 'block' ... or, the templatedata api has a bug (I thought this bug was fixed way back as part of T128337 .. but maybe it resurfaced?)

Deskana, I don't know what you mean by by block formatting. Is it considered block formatting when the whitespace gets stripped out of Template:Infobox film here?: https://en.wikipedia.org/w/index.php?title=Kabali_(film)&diff=808234431&oldid=808184842. Also, please note that this technical stuff is absolutely not my speciality, nor is it in any way intuitive for me, so I really don't know what to do here.

From what I understand, there's no Enwiki global policy or guideline about how any of these templates should be presented. Some editors (like me) prefer lining up the equals signs.

director = Sally Roe
cinematographer = John Doe
editor = Dave McBoatface

Some editors prefer

director = Sally Roe
cinematographer = John Doe
editor = Dave McBoatface

(I can't make the extra spaces appear, for I do not understand phabricator markup yet. But if you look at the raw text, you'll see what I mean)

But unless a clear consensus has been established at (for example) Template talk:Infobox Film for how template parameters should be indented (which in most cases I would assume has never been done), then Visual Editor is basically silently overriding any local consensus that's been achieved through editing. That seems disruptive to me. Thanks all.

I don't get it. why are we looking at the specified block format for anything other than inserting a template or a template param from scratch ? I think we learned by now that making changes to what is already present in the wikicode is always heavily frowned upon by the wikicode purists ?

@Cyphoidbomb block formatting, is where it's preferred for a template to start on it's own line, with attributes mostly on new lines as well, as opposed to inline, which is basically {{this}} smack in the middle of a sentence. Since VisualEditor isn't going to ask every single editor how a piece of wikicode that they don't understand is supposed to 'look', it makes assumptions, or you have to tell it. Here is was told by [[Template:Infobox television]], that it should use block formatting.

Deskana, I don't know what you mean by by block formatting. Is it considered block formatting when the whitespace gets stripped out of Template:Infobox film here?: https://en.wikipedia.org/w/index.php?title=Kabali_(film)&diff=808234431&oldid=808184842. Also, please note that this technical stuff is absolutely not my speciality, nor is it in any way intuitive for me, so I really don't know what to do here.

Yes, that is block format. Block format does not have additional whitespace between the parameter name and equals sign.

But unless a clear consensus has been established at (for example) Template talk:Infobox Film for how template parameters should be indented (which in most cases I would assume has never been done), then Visual Editor is basically silently overriding any local consensus that's been achieved through editing. That seems disruptive to me. Thanks all.

Then you should speak to the editor instructed Parsoid to do this, thereby violating consensus. Some editor, at some point, instructed Parsoid to do this by adding "format": "block" to the template data at Template:Infoboxfilm/doc. Parsoid is doing exactly what that editor told it to do, which is use block formatting.

If you wish to change this instruction, you can go to that page, and where it says

 },
 "format": "block"
}

you can replace it with

 }
}

I don't get it. why are we looking at the specified block format for anything other than inserting a template or a template param from scratch ? I think we learned by now that making changes to what is already present in the wikicode is always heavily frowned upon by the wikicode purists ?

@cscott may wish to comment on this as the author of that patch.

Usually i try to filter stuff, but once in a while it's good to just copy and paste the vitriol in here to better reflect how people feel about things like this.

I don't even have to check the archives to know this must be the 100th complaint about this. There are many things to despise about Visual Editor and the way it was designed and developed, but surely in the top 10 is the fact that, when a template is edited, it reorders all the parameters so that diffs are impossible to interpret. [1] I don't want to hear about how technically challenging it would be to fix this. Just fix it, or disable editing of templates in VE, or something. EEng 18:02, 31 October 2017 (UTC)

It's either a bug or a feature depending on your point of view. What it does do is bring consistency to the random orderings of fields created by users, which is quite annoying too. Personally I like to see infobox fields in the same sequence so I don't actually mind the result. But I agree that the first time it happens it makes a messy diff. This is because we don't have a very good diff tool. Indeed, if you want to be sneaky about a small change, you make it and then switch some paragraphs around. The human reader of the diff will tend not to notice the small content change in the presence of the larger re-ordering change. Some diff tools can show moved-but-unchanged information differently, so it is clearer what is a re-ordering of content vs what is a change in content. If our diff tool could make this distinction, it would be less of a problem not just for this VE re-ordering but for spotting those sneaky edits too. Kerry (talk) 04:12, 1 November 2017 (UTC)

Well, we don't have that diff tool, and the parameters aren't in a "random ordering", but rather the order the article's editors put them in for various reasons. So it's a bug. I'd appreciate hearing from the geniuses who designed it this way WTF they thought they were doing. EEng 04:58, 1 November 2017 (UTC)

I believe that the Visual Editor is maintaining the the fields in the same order as the template definition. I believe work is in progress on a better diff tool; it was presented at Wikimania in August. You can find the slides and a video linked off here. Frankly there is a lot to be said for actually trying the Visual Editor. I was quite fluent in the source editor, but I find that there are many things are done better and more easily in the Visual Editor and, since you can switch between them, even within a single edit, you can have the best of both worlds. Kerry (talk) 07:15, 1 November 2017 (UTC)

I don't know if the VE is still organizing parameter out of sequence which was previously the problem I'd noticed about this, but yes, it was annoying. @EEng:, you might consider going into Preferences > Gadgets, and activating wikEdDiff. While viewing a diff, you'll have the option of clicking a little triangle doohickey (Delta?) and a more intuitive drop-down will help you make sense of the changes that were made. Regards, Cyphoidbomb (talk) 14:40, 2 November 2017 (UTC)

What isn't touched, should not change. It's very simple. We've know this for ages, and we keep mucking it up and then telling people they shouldn't care. It doesn't work like that.

There seems to be some inter-editor tension here. Some editors definitely prefer a certain field ordering and argument layout (like Kerry above). Those folks encode those preferences in the templatedata for a given template. Other folks then see these requested preferences applied by VisualEditor and blame the tool rather than their fellow editor.

That said, there was some refactoring to how Parsoid handles templatedata which made it respect the editor-supplied templatedata preferences in more cases. We can consider adding back some complexity to Parsoid to try to tread the fine line between "wikitext as it is" and "wikitext as we've been told it should be". I'm not certain this is going to do anything about editor anger, though -- since the folks who authored the templatedata preferences in the first place would probably complain that VE is then letting arguments be in the wrong order, or with a confusing mishmash of parameter styles, etc.

As Kerry mentioned, this is generally a problem only the first time a given template is touched. After that, the arguments will match the templatedata-requested format and future diffs will be clean.

Perhaps the editor community who works on creating templatedata specifications could also write a bot which proactively normalizes templates to these specifications? That would prevent VE from being responsible for the eventual dirty diff. (In fact the "custom" field format feature of TemplateData was inspired by a request from @thiemowmde at Wikimania, who has exactly this sort of bot; see T138492.)

Another middle ground might be to always respect the original argument order (ignoring templatedata), while still honoring the per-argument formatting specs. This would alleviate some of the issues resulting from our current line-oriented diff mechanisms, although we'd probably still have whitespace-related complaints.

Deferring to product and the liaisons to chart a path forward.

cscott, you completely miss the point. There is no "inter-editor tension". There may very well be "Some editors definitely prefer a certain field ordering and argument layout", but on enwp at least, such editors are not allowed to impose that preference on articles at will, and certainly are not allowed to use a "the tool made me do it" excuse.

The confusing diffs are not the real problem, and I blame myself for making it seem that way in my post quoted in a post a bit up from here. The real problem is that you're reordering stuff that was put in a certain order, for reasons you don't know, by article editors. For example, a group of parameters in an infobox may be grouped together, with a comment explaining something about them as a group (e.g. resolving a conflict of sources). By juggling the order you make nonsense of these comments.

I'm fascinated (and a bit disturbed) by this concept of "wikitext as we've been told it should be". Who's told you that part of wikitext as it "should be" is some rigid order and formatting of template parms?

You idea that "Perhaps the editor community who works on creating templatedata specifications could also write a bot which proactively normalizes templates to these specifications? That would prevent VE from being responsible for the eventual dirty diff" shows a complete misunderstanding of the how things are done in the field. No bot like this would ever be approved. Part of the STRENGTH of templates is that parms can be presented in any order.

Looks like an editor removed the offending block-formatting directive from this particular infobox -- so going forward, edits to transclusions of this template should leave things untouched.

Another middle ground might be to always respect the original argument order (ignoring templatedata), while still honoring the per-argument formatting specs. This would alleviate some of the issues resulting from our current line-oriented diff mechanisms, although we'd probably still have whitespace-related complaints.

If people are intentionally using parameters in a specific order for whatever reason, then it would be good if Parsoid could respect that.

You idea that "Perhaps the editor community who works on creating templatedata specifications could also write a bot which proactively normalizes templates to these specifications? That would prevent VE from being responsible for the eventual dirty diff" shows a complete misunderstanding of the how things are done in the field. No bot like this would ever be approved. Part of the STRENGTH of templates is that parms can be presented in any order.

@EEng please see T138492, which contains just such a bot, used on dewiki. I don't mean to inflame tensions here, I'm just noting that we (engineering) are being pulled in different directions by different segments of the editor community. I have no particular dog in this fight, but I get to be yelled at by both sides when the other side gets its way. It's okay, that comes with the job.

(I'll note in passing that enwiki and dewiki typically have disagreements based on the benefit of rigidly adhering to particular forms or rules or styles, but this might not be immediately obvious to those who work primarily on one wiki or the other.)

If people are intentionally using parameters in a specific order for whatever reason, then it would be good if Parsoid could respect that.

That seems like a sensible middle ground in this case, although there's a lot of wiggle room in that "intentionally using" part.

No doubt the Germanic love of order and regimentation plays a role in that. More seriously... I should have been more specific: no such bot would ever be approved on enwiki. If you want you can have two modes for VE to operate in (reorder, don't reorder), or a button/checkbox you can select as you exit editing of a particular template. But as long as you continue to force the reordering of parameters like this, it'll be just one more reason VE isn't being accepted (on enwiki, at least).

A lot of the discussion here is appears to be missing the point. As it see it, the issue is fairly simple:

Dirty diffs are bad in general. In this case, scrambling the order of existing template parameters is an especially disruptive dirty diff.

(1) If parameters are grouped or ordered for a good reason, every time VE shows up it will scramble that work. Any sane person can only quit in frustration when VE intransigently edit wars the order.
(2) VE makes the user appear to be potential vandal. A lot of slow painful mental labor needs to be expended to examine the diff in detail, attempting to determine whether the scrambling is camouflaging a malicious change. (Or hiding a good-faith-but-harmful change.) Not only does this a waste editor-time, editors become frustrated when they realize their effort was wasted. That frustration gets directed against VE and/or against the WMF. This issue has been reported countless times. The WMF has persistently dismissed those reports, which has also been raising frustrations.

This isn't a hard task. Just remember and preserve the existing order. The "messy" case is how to add new parameter(s) to existing parameters. If the existing order matches the default order, this is trivial. If not, I offer this heuristic: If a new parameter is adjacent to an existing parameter in the default-ordering, insert the new parameter adjacent to that existing parameter. When you run out of adjacencies, insert one parameter using the closest non-adjacent pairing. Repeat until done. If you could equally insert after X or insert before Y, a new parameter prefers to be inserted next to another new parameter.

Another middle ground might be to always respect the original argument order (ignoring templatedata), while still honoring the per-argument formatting specs. This would alleviate some of the issues resulting from our current line-oriented diff mechanisms, although we'd probably still have whitespace-related complaints.

If people are intentionally using parameters in a specific order for whatever reason, then it would be good if Parsoid could respect that.

https://gerrit.wikimedia.org/r/#/c/389852/

If I can summarize what I think to be an important frame of mind: Visual Editor is not the Auto-Wiki Browser, and it probably shouldn't be used that way.

(From discussion I had on MediaWiki:)
:Per MOS:VAR: "On some questions of style, the MoS proposes more than one acceptable answer; on other questions it gives no guidance. The Arbitration Committee has ruled that editors should not change an article from one styling to another without 'substantial reason'"
No substantial reason has been put forth for why skill-less editors are being used as botfarms to obliviously make silent, bold formatting changes that may contravene local consensus or the status quo. Thus, I argue, Visual Editor should stop messing with stuff.

Anyway, energetic exaggerations aside, if editors are unaware of the changes they are making, they can neither seek consensus beforehand, nor are they likely to be able to argue for why they made those changes. They're basically shills. Something to consider?

I believe one of the regulars at en:WikiProject Television and en:WikiProject Film was able to manipulate the two infobox templates so that maybe this won't be an issue for those, but this is a problem elsewhere, like here.

(Note: I originally started writing this post weeks ago, but I guess I forgot to submit. If anything doesn't make any sense, it's because it's mostly an old post.)

This problem seems significantly less annoying now with param order preservation !
Thanks @Arlolra !

Thanks @Arlolra !

All I did was review and deploy it. @ssastry and @cscott wrote the patch. So, thanks goes to them.

Arlolra claimed this task.

Tentatively going to try closing this.

Thanks to the Parsing folks for this fix. :-)

I'm almost afraid to ask -- what does that mean?

The point value? Zero points means that the fix took none of the VisualEditor devs' own time.

The parameter reordering issue was addressed (yeay), but the original task here was to stop stripping whitespace. Here's a test edit showing that the problem still exists.

The parameter reordering issue was addressed (yeay), but the original task here was to stop stripping whitespace. Here's a test edit showing that the problem still exists.

That's requested by the format=block parameter of the template's TemplateData: https://en.wikipedia.org/wiki/Template:Infobox_organization/doc?veaction=editsource&section=4

That's not the only unusual thing in the templatedata, many of the parameters have unusual trailing whitespace. :(

If you want aligned parameters, Ed mentioned the correct solution:

The formatting for 'Infobox television' is 'unspecified', which I assume means leave whitespace alone, but the Parsoid team will need to answer this.

There is currently no an automated setting for space-aligned parameters.

I've been corrected below, you can use the following format:
{{_\n|_______________ = _\n}}\n

(with as many underscores as spaces are required)

@cscott, thanks but unhelpful. What templatedata value should I use to tell VE not to mangle existing content? If there's an answer I will happily close this task as resolved, and I will open a community process to convert every templatedata to use that value.

As TheDJ tried to explain, as I tried to explain, we have all known for years that generating dirty diffs is considered a bug. Unsurprisingly, you are getting a stream of requests for this bug be fixed.

If Bob's infobox has aligned parameters and born 1942, and Rob's infobox has unaligned parameters and born 1942, and I edit each from 1942 to 1943, they should both be one-character diffs. There's no reason for VE to touch anything before the existing = sign. And more importantly VE must not modify unchanged parameters. If you want to be super-duper-awesome, VE could make a best-effort attempt to match existing style when adding new parameters. That would be ideal, that is what a human would do. However that would be a bonus.

@Alsee in this case, if you had entries which were whitespace aligned, whichever editor put 'format:block' in the templatedata was at fault. Looking at the edit you cited, it appears as if the format should have been:

{{_\n| _____________ = _\n}}\n

Note that if an incorrect format string is used, new arguments will be added with formatting inconsistent with existing entries, so that would lead to complaints as well. The ideal solution is to use the correct format string which matches existing uses. If the existing uses are a mix of different argument formatting styles, it will likely to be impossible to make everyone happy all the time.

@cscott you suggested two options. One, you yet-again proposed "custom format string", which we have told you DOESN'T FIX ANYTHING. That will still mangle existing parameters.

The other suggestion was to remove this templatedata value. I just want to be clear if this is the fix you are proposing. I will happily go to the community an open a proposal to DELETE AND BAN these values from being used on any template. However fiddling with templatedata isn't one of my areas of expertise, and I'd rather not waste my time (and waste the community's time) on your suggested fix if there's some problem with it. I don't want to waste time if I'm just going to wind up back here with a community-consensus saying the software needs to be fixed.

So, are you declining this task on the basis that the community should fix it by deleting these values from all of our templates? And if so, why create them in the first place? Why not just yank them from the code completely, so we don't have to worry about someone adding one of these disruptive values in the future?

As TheDJ tried to explain, as I tried to explain, we have all known for years that generating dirty diffs is considered a bug. Unsurprisingly, you are getting a stream of requests for this bug be fixed.

If Bob's infobox has aligned parameters and born 1942, and Rob's infobox has unaligned parameters and born 1942, and I edit each from 1942 to 1943, they should both be one-character diffs. There's no reason for VE to touch anything before the existing = sign. And more importantly VE must not modify unchanged parameters.

Overall, Parsoid will not be able to avoid all dirty diffs completely everywhere - there will be some normalizations occasionally. We can always add additional code to handle more dirty diff scenarios. I think one way to evaluate whether it is worth doing it or not is by determining how common are these dirty diffs -- i.e. if most templates are complying with the templatedata format, then the dirty diffs will be uncommon for only the non-compliant ones.

I think @cscott was only suggesting a different format in case it reduces how many dirty diffs there are.

Overall, there is a challenge for us as developers to balance code complexity (with a view of keeping maintenance burden under control) with feature requests. In this case, it felt like a reasonable trade-off to reduce code complexity since it will also gradually get more transclusions comply with the editor-desired formatting for the template. It seem like it would be a long-term benefit that accumulates incrementally.

For example, with the ordering of template parameters, once we had better clarity about what the issue was, we made the necessary fix even if it added more code and complexity (and in which you and others participated in as well). But, it isn't clear how much code and complexity we should add to tackle this problem. So, if you could help us understand how common / uncommon this dirty diff-ing is, it can better inform the kind of tradeoffs we can / should make in Parsoid.

@ssastry I was hoping it would be sufficient to cite the known issue of dirty diffs and the fact that people clearly want this fixed. I previously drafted-but-cut a longer explanation. I wanted to avoid posting a rantish wall-of-text. But ok. Maybe better understanding will help. Let's start here:

it felt like a reasonable trade-off to reduce code complexity since it will also gradually get more transclusions comply with the editor-desired formatting for the template.

If someone is using VE to add a template, and the WMF is not allowing that person to see or control the formatting, it's not unreasonable for a random individual editor to add a suggested-format to templatedata. However that individual absolutely does not have authority to impose a Wikipedia-wide bot-run to forcibly covert everyone else's pre-existing work to their preferred style. Editors at those pre-existing articles may have a good reason for a different style, and in some cases it's simply a matter of difference of opinion on which is the best/preferred style.

So no, you're not converting existing templates to "editor-desired formatting for the template". We do not buy the WMF's attempt to place blame on the person who sets a default style for new templates added via VE. Maybe you think it's a swell idea to "normalize" existing templates. Maybe you felt dirty diffs were the quickest and most convenient thing to implement. However either way it still means that the WMF and only the WMF made the content-decision to edit that part of my work.

It's even worse if editors at an article have used some style for a reason. VE behaves as an intransigent bot repeatedly and disruptively editwarring every time it shows up on that page. VE's persistent behavior means that any human editor can only surrender-the-editwar to VE in frustration, or quit-the-project in frustration.

Any human who behaved as VE is behaving would be blocked for tendentious editing. Anyone who ran a bot like this would be blocked for unapproved project wide rewrites. The closest comparison would be having AutoWikiBrowser make these kinds of changes. That would receive a similarly hostile reception.

An additional issue is dirty diffs themselves. When VE was scrambling parameter order, it required very slow and painstaking mental labor to compare a scrambled diff to figure out what did or did not change. A whitespace dirty diff is a smaller version of the same thing. It results in a needlessly longer diff, and that longer diff still needs to be scanned for understanding. It consumes a small but non-zero mental capacity to identify and dismiss a pagefull of pointlessly distracting irrelevant changes. That expands when conscious attention is wasted on active annoyance at the pointless nuisance. That annoyance winds up directed at VE itself. The WMF tends to be rather eager on anything to increase acceptance of VE / to diminish hostility to VE. On that basis alone the WMF should be eager to address cases of dirty diffs. They're an annoying VE thorn that keeps poking us, regardless of whether we use VE or not.

I've tried not to get too ranty and I've left out some tangential frustration. But in short the WMF's aggressive disdain for wikitext, and constant pushback on things we want and need, doesn't help.

Please note that this bug is causing some issues with AbuseFilter, because it's adding lines which are not really modified to the modified lines category. For instance, this change wouldn't have triggered the filter without this bug.

If it is that important for some communities to have the current behavior, please make it a feature (by adding, for instance, a "cleanup" option to templateData. The default should be to leave the wikitext untouched.

Not sure if this is related or a new separate issue but I'm now noticing edits that remove all the white-space including newlines that makes determining what has been changed very difficult. This makes editing after with the source editor awful, but more importantly makes vandalism detection much harder.

Found example: https://en.wikipedia.org/w/index.php?title=Economy_of_Serbia&diff=878557234&oldid=878477219
My test example: https://en.wikipedia.org/w/index.php?title=User:KylieTastic/sandbox&diff=878559940&oldid=878559771

Even in case thers's no format value in TemplateData (or no TemplateData / no doc for given tamplate at all), VE should not change block template markup into inline (and vice-versa). It should not be that hard to infer inline/block format from actual markup during parsing (witespace containing newline before/after argument separators, maybe some simple RE based heuristics).

Not sure if this is related or a new separate issue but I'm now noticing edits that remove all the white-space including newlines that makes determining what has been changed very difficult. This makes editing after with the source editor awful, but more importantly makes vandalism detection much harder.

This recent issue (with all spaces disappearing, even for templates with format:block or format:null) seems to be separate, and is now tracked as T213922.

(...) For example, a group of parameters in an infobox may be grouped together, with a comment explaining something about them as a group (e.g. resolving a conflict of sources). By juggling the order you make nonsense of these comments.

I think the behaviour is not optimal for dealing with comments between comments. See e.g. this edit:
https://pt.wikipedia.org/w/index.php?diff=54180650
Instead of starting a new line for the parameter "| nome ", this is after a comment, in the same line.

OK, so maybe that particular practice isn't a good idea. But there are lots things editors do to group and format parameters that ARE sensible, so automation shouldn't be imposing its own ideas.

Besides the annoyance of the bug itself, it seems to me that it drafts wikitext editors into involuntary servitude as housekeepers to sweep up after the actions of good-faith VE editors trying to improve an article, who are thwarted by the actions of a tool through no fault of their own, and who are prevented from carrying out a workaround by the tool. To me, this is the greater problem.

Raised at mw:VisualEditor/Feedback#Does VE munge white space? (Permalink).

We'll revisit this in the coming week or two and see what is going on.

prevented from carrying out a workaround by the tool.

There are various workarounds. The first is to fix the format, which I have done for the template in question (https://w.wiki/RaE). If the community prefers to not have any format imposed by the template, then the format block can be removed completely as described in T179259#3730002.