rendering errors on formulas

<math>\text{rot}\vec E = \dot \vec B=0</math>

Attached:

There are rendering errors in formulars. This seems also a problem of nesting.

The bug is also happening on Portuguese Wikipedia, on pages which were not changed in the last few days. E.g.:
https://pt.wikipedia.org/w/index.php?diff=27158224&oldid=26994357&uselang=en#Representa.C3.A7.C3.A3o_alg.C3.A9brica

If this bug is going to persist for a while, it might be nice to address the enhancement request in bug 31546. Specifically, converting the treatment of Math error messages to wikitext (from plain text) so that we can add tracking categories to help identify which pages are having problems. That way we can deploy work-arounds if we have to. Right now there is no easy way of knowing how widespread the failed renderings are, though it is seems likely that they are effecting many thousands of math and science pages.

(In reply to comment #4)

Right now there is no easy way of knowing
how widespread the failed renderings are, though it is seems likely that they
are effecting many thousands of math and science pages.

A few of them are indexed by Google:
http://www.google.com.br/search?q=%22Failed+to+parse+(PNG+conversion+failed%22+site%3Awikipedia.org

I tried rendering a bunch of variants of the example in comment 2 on en.wikipedia.org; no visible problems.

All the pages I see in the top Google results from link in comment 5 render fine for me, except the one that explicitly tries a bunch of raw Unicode characters which aren't supported as literals.

Possibly the updated texvc was simply missing on a bunch of machines, or /math output dir wasn't mounted on some machines, and it's since been resolved either manually or through regular processes like puppet updates.

(In reply to comment #6)

I tried rendering a bunch of variants of the example in comment 2 on
en.wikipedia.org; no visible problems.

All the pages I see in the top Google results from link in comment 5 render
fine for me, except the one that explicitly tries a bunch of raw Unicode
characters which aren't supported as literals.

Possibly the updated texvc was simply missing on a bunch of machines, or /math
output dir wasn't mounted on some machines, and it's since been resolved either
manually or through regular processes like puppet updates.

Dozens (hundreds?) of pages have already been updated using the work arounds.

For examples of math code that used to work and now fails, look at old versions of pages like:

https://en.wikipedia.org/w/index.php?title=Orientation_(computer_vision)&oldid=430039796

http://en.wikipedia.org/w/index.php?title=Mellin_transform&oldid=428975706

Also, the example given by Biezl in Comment 1 is a special case. It doesn't generate an error. Instead in renders a qualitatively different result than prior to the update. In that example, the vector arrow now appears to the upper left of the B rather than on top of it as it did prior to the update.

(In reply to comment #7)

Dozens (hundreds?) of pages have already been updated using the work arounds.

Indeed. There was some discussion about this at [[Wikipedia_talk:WikiProject_Mathematics#TeX_not_rendering]]

Excellent --- concrete examples are VERY helpful here, and will help us to confirm whether this is for instance caused by changes to whitespace parsing.

I've found some circumstances under which improper (misplaced) rendering happens.

Code is tested on de.wikipedia.org and en.wikipedia.org

1. <math>\dot \vec B</math> not rendering
2. <math>\dot {\vec B}</math> proper rendering
3. <math>\dot \vec B = X</math> not rendering
4. <math>\dot \vec B = \text{X}</math> improper rendering
5. <math>\dot \vec B = \mathrm X</math> improper rendering

I can't reproduce errors with those examples.

Most servers are running a texvc with the same crc32 checksum.

The following host have a different checksum:
srv191
srv193
srv192
srv290
srv292
srv293
srv294
srv295
srv296
srv297
srv298
srv299
srv300
srv301
snapshot3
...they all the same checksum (but different than the others).

Permissions look OK on all servers.

(In reply to comment #10)
I've tested my TEX code on a diffrent MediaWiki (1.15) and recognized slightly diffrent rendering between case 1-3 and 4-5.

I've tested Mediawiki 1.18 with preview, saved article, with login and without on FireFox 7, Opera 11.50 and Chrome 14. Tested sites include now Commons, MediaWiki.org and Incubator with preview. Don't know if texvc reacts on browser?

BTW: Some people decided to patch all tex code in wikipedia, since I could only find few broken articles.

nmichalo wrote:

Explanation of Parser behavior.

The parser previously added many braces (aka { } ) around expressions generously. This was causing issues with commands that produced space and was breaking some commands. For example It would translate <math>\operatername{sen} x</math> to $${\operatorname {sen}}x$$ does not produce the correct result.

As a result the extra braces were removed. This causes strange behavior relating to math accents in the following ways:

1. Code that would used to compile under LaTeX is no longer sanitized to work under texvc. Examples include 1 from comment #10.

2. The amsmath package changes the way accents are handled (as discussed in their user documentation). Texvc only loads amsmath as needed. Which means if a command such as \text appears then the behavior suddenly seems to change.

This is why Which is why <math>\dot\vec B</math> will not render but <math>\text{rot}\dot\vec B</math> will render, but incorrectly.

Technically speaking amsmath insists all math accents use braces on all math accents. Thus despite the example the example of <math>\text{rot}\dot{\vec{B}}</math> actually compiling other examples do not compile exactly because of amsmath such as <math>\tilde \mathcal{M}</math> which is what was happening at the Mellin Transform page. This code $$\tilde \mathcal{M}$$ is fine as vanilla LaTeX but not accepted by amsmath.

I am testing a patch now which does the following:

1. Restores the original placement of braces of braces.
2. Introduces a new function specifically to handle commands which are brace sensitive such as \operatorname.
3. Removes some unneeded white space for the sanitized LaTeX.

I could use the following information/examples if anyone comes across them:
Potentially many other examples of incorrect LaTeX were sanitized into working by way of adding braces. Do we know of any examples of things that broke that do not involve a math accent such as \dot, \vec, \tilde, \hat, etc.?

I am interested because I think it would be wise to document the ways in which in which texvc extends/differs from LaTeX. There is, and has been for a long time interest in moving away from texvc to blahtex, mathjax, etc. And if we cannot give guidelines as to the non-standard LaTeX occuring within the wikimedia sites, it could be a rough transition.

nmichalo wrote:

Uploaded patch r99741.

tagging bugs for Marcus to look at

lowering priority since this can be fixed on-wiki.

It'd be good to include a pointer to how to fix this. Otherwise, not a blocker.

nmichalo wrote:

By fix this do yo mean fix texvc or fix correct the wikicode?

To fix the wikicode: Ensure all decorations \dot, \vec, \tilde, \hat, etc, and font chagnes (\mathcal ) place curly braces {} around their arguments. Depending on the circumstances this is not strictly speaking always required, but it will always fix any problems.

Examples

 Bad LaTeX:                   Fixed:
\tilde\mathcal f             \tilde{\mathcal{f}}
\dot \vec B                  \dot{\vec{B}}

But when upgrading this could potentially break a lot of formulas. But I defer to your judgment about what is or is not a blocker.

Isn't the r99741 the solution, is it? It needs deployment on the WMF cluster.

removing dep on tarball blockers for things that aren't in mw core.

Can someone review r99741 and if it is OK, deploy it?

The change was deployed on live site by Sam Reed some minutes ago.

I have added some snippets from above comment on the enwiki page:
https://en.wikipedia.org/wiki/User:Hashar/bug31442

There are still some wrong rendering but overall case that were not rendering do render now.

I am considering this bug fixed now.