Math not rendering
Closed, ResolvedPublic

Assigned To
None
Priority
Normal
Author
bzimport
Subscribers
brion, He7d3r, hashar and 5 others
Projects
Reference
bz31442
Description

Author: rohde

Description:
Following the 1.18 deployment, there are now many examples of <math> expressions that previously rendered correctly and now report the error:

"Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert))"

Simple examples include:

<math>\dot \vec B</math>

<math>\chi(\hat\mathbf{C})=2.</math>

A work around exists by adding extra braces to the TeX markup as described here:

http://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#Math.2FLatex

However, it shouldn't really be necessary to go around and update tons of math expressions that were working before the upgrade.


Version: unspecified
Severity: major

bzimport added a project: Math.Via ConduitNov 21 2014, 11:51 PM
bzimport added a subscriber: wikibugs-l.
bzimport set Reference to bz31442.
bzimport created this task.Via LegacyOct 6 2011, 10:49 PM
Menner added a comment.Via ConduitOct 7 2011, 8:55 AM

rendering errors on formulas

<math>\text{rot}\vec E = \dot \vec B=0</math>

Attached: url.txt

Menner added a comment.Via ConduitOct 7 2011, 8:57 AM

There are rendering errors in formulars. This seems also a problem of nesting.

He7d3r added a comment.Via ConduitOct 7 2011, 10:46 AM

The bug is also happening on Portuguese Wikipedia, on pages which were not changed in the last few days. E.g.:
https://pt.wikipedia.org/w/index.php?diff=27158224&oldid=26994357&uselang=en#Representa.C3.A7.C3.A3o_alg.C3.A9brica

Dragons_flight added a comment.Via ConduitOct 9 2011, 12:57 AM

rohde wrote:

If this bug is going to persist for a while, it might be nice to address the enhancement request in bug 31546. Specifically, converting the treatment of Math error messages to wikitext (from plain text) so that we can add tracking categories to help identify which pages are having problems. That way we can deploy work-arounds if we have to. Right now there is no easy way of knowing how widespread the failed renderings are, though it is seems likely that they are effecting many thousands of math and science pages.

He7d3r added a comment.Via ConduitOct 9 2011, 1:09 AM

(In reply to comment #4)

Right now there is no easy way of knowing
how widespread the failed renderings are, though it is seems likely that they
are effecting many thousands of math and science pages.

A few of them are indexed by Google:
http://www.google.com.br/search?q=%22Failed+to+parse+(PNG+conversion+failed%22+site%3Awikipedia.org

brion added a comment.Via ConduitOct 10 2011, 8:06 PM

I tried rendering a bunch of variants of the example in comment 2 on en.wikipedia.org; no visible problems.

All the pages I see in the top Google results from link in comment 5 render fine for me, except the one that explicitly tries a bunch of raw Unicode characters which aren't supported as literals.

Possibly the updated texvc was simply missing on a bunch of machines, or /math output dir wasn't mounted on some machines, and it's since been resolved either manually or through regular processes like puppet updates.

Dragons_flight added a comment.Via ConduitOct 10 2011, 8:18 PM

rohde wrote:

(In reply to comment #6)

I tried rendering a bunch of variants of the example in comment 2 on
en.wikipedia.org; no visible problems.

All the pages I see in the top Google results from link in comment 5 render
fine for me, except the one that explicitly tries a bunch of raw Unicode
characters which aren't supported as literals.

Possibly the updated texvc was simply missing on a bunch of machines, or /math
output dir wasn't mounted on some machines, and it's since been resolved either
manually or through regular processes like puppet updates.

Dozens (hundreds?) of pages have already been updated using the work arounds.

For examples of math code that used to work and now fails, look at old versions of pages like:

https://en.wikipedia.org/w/index.php?title=Orientation_(computer_vision)&oldid=430039796

http://en.wikipedia.org/w/index.php?title=Mellin_transform&oldid=428975706

Also, the example given by Biezl in Comment 1 is a special case. It doesn't generate an error. Instead in renders a qualitatively different result than prior to the update. In that example, the vector arrow now appears to the upper left of the B rather than on top of it as it did prior to the update.

He7d3r added a comment.Via ConduitOct 10 2011, 9:16 PM

(In reply to comment #7)

Dozens (hundreds?) of pages have already been updated using the work arounds.

Indeed. There was some discussion about this at [[Wikipedia_talk:WikiProject_Mathematics#TeX_not_rendering]]

brion added a comment.Via ConduitOct 10 2011, 10:02 PM

Excellent --- concrete examples are VERY helpful here, and will help us to confirm whether this is for instance caused by changes to whitespace parsing.

Menner added a comment.Via ConduitOct 11 2011, 7:30 AM

I've found some circumstances under which improper (misplaced) rendering happens.

Code is tested on de.wikipedia.org and en.wikipedia.org

  1. <math>\dot \vec B</math> not rendering
  2. <math>\dot {\vec B}</math> proper rendering
  3. <math>\dot \vec B = X</math> not rendering
  4. <math>\dot \vec B = \text{X}</math> improper rendering
  5. <math>\dot \vec B = \mathrm X</math> improper rendering
aaron added a comment.Via ConduitOct 11 2011, 5:43 PM

I can't reproduce errors with those examples.

aaron added a comment.Via ConduitOct 11 2011, 5:55 PM

Most servers are running a texvc with the same crc32 checksum.

The following host have a different checksum:
srv191
srv193
srv192
srv290
srv292
srv293
srv294
srv295
srv296
srv297
srv298
srv299
srv300
srv301
snapshot3
...they all the same checksum (but different than the others).

Permissions look OK on all servers.

Menner added a comment.Via ConduitOct 11 2011, 6:55 PM

(In reply to comment #10)
I've tested my TEX code on a diffrent MediaWiki (1.15) and recognized slightly diffrent rendering between case 1-3 and 4-5.

I've tested Mediawiki 1.18 with preview, saved article, with login and without on FireFox 7, Opera 11.50 and Chrome 14. Tested sites include now Commons, MediaWiki.org and Incubator with preview. Don't know if texvc reacts on browser?

BTW: Some people decided to patch all tex code in wikipedia, since I could only find few broken articles.

bzimport added a comment.Via ConduitOct 13 2011, 4:39 PM

nmichalo wrote:

Explanation of Parser behavior.

The parser previously added many braces (aka { } ) around expressions generously. This was causing issues with commands that produced space and was breaking some commands. For example It would translate <math>\operatername{sen} x</math> to $${\operatorname {sen}}x$$ does not produce the correct result.

As a result the extra braces were removed. This causes strange behavior relating to math accents in the following ways:

  1. Code that would used to compile under LaTeX is no longer sanitized to work under texvc. Examples include 1 from comment #10.
  1. The amsmath package changes the way accents are handled (as discussed in their user documentation). Texvc only loads amsmath as needed. Which means if a command such as \text appears then the behavior suddenly seems to change.

This is why Which is why <math>\dot\vec B</math> will not render but <math>\text{rot}\dot\vec B</math> will render, but incorrectly.

Technically speaking amsmath insists all math accents use braces on all math accents. Thus despite the example the example of <math>\text{rot}\dot{\vec{B}}</math> actually compiling other examples do not compile exactly because of amsmath such as <math>\tilde \mathcal{M}</math> which is what was happening at the Mellin Transform page. This code $$\tilde \mathcal{M}$$ is fine as vanilla LaTeX but not accepted by amsmath.

I am testing a patch now which does the following:

  1. Restores the original placement of braces of braces.
  2. Introduces a new function specifically to handle commands which are brace sensitive such as \operatorname.
  3. Removes some unneeded white space for the sanitized LaTeX.

I could use the following information/examples if anyone comes across them:
Potentially many other examples of incorrect LaTeX were sanitized into working by way of adding braces. Do we know of any examples of things that broke that do not involve a math accent such as \dot, \vec, \tilde, \hat, etc.?

I am interested because I think it would be wise to document the ways in which in which texvc extends/differs from LaTeX. There is, and has been for a long time interest in moving away from texvc to blahtex, mathjax, etc. And if we cannot give guidelines as to the non-standard LaTeX occuring within the wikimedia sites, it could be a rough transition.

bzimport added a comment.Via ConduitOct 13 2011, 11:06 PM

nmichalo wrote:

Uploaded patch r99741.

MarkAHershberger added a comment.Via ConduitOct 15 2011, 10:02 PM

tagging bugs for Marcus to look at

MarkAHershberger added a comment.Via ConduitOct 25 2011, 11:13 PM

lowering priority since this can be fixed on-wiki.

MarkAHershberger added a comment.Via ConduitOct 28 2011, 2:07 AM

It'd be good to include a pointer to how to fix this. Otherwise, not a blocker.

bzimport added a comment.Via ConduitOct 28 2011, 3:46 AM

nmichalo wrote:

By fix this do yo mean fix texvc or fix correct the wikicode?

To fix the wikicode: Ensure all decorations \dot, \vec, \tilde, \hat, etc, and font chagnes (\mathcal ) place curly braces {} around their arguments. Depending on the circumstances this is not strictly speaking always required, but it will always fix any problems.

Examples

 Bad LaTeX:                   Fixed:
\tilde\mathcal f             \tilde{\mathcal{f}}
\dot \vec B                  \dot{\vec{B}}

But when upgrading this could potentially break a lot of formulas. But I defer to your judgment about what is or is not a blocker.

Raymond added a comment.Via ConduitOct 28 2011, 5:55 AM

Isn't the r99741 the solution, is it? It needs deployment on the WMF cluster.

MarkAHershberger added a comment.Via ConduitOct 28 2011, 2:41 PM

removing dep on tarball blockers for things that aren't in mw core.

Strainu added a comment.Via ConduitNov 10 2011, 8:43 AM

Can someone review r99741 and if it is OK, deploy it?

hashar added a comment.Via ConduitNov 15 2011, 10:33 PM

The change was deployed on live site by Sam Reed some minutes ago.

I have added some snippets from above comment on the enwiki page:
https://en.wikipedia.org/wiki/User:Hashar/bug31442

There are still some wrong rendering but overall case that were not rendering do render now.

I am considering this bug fixed now.

Add Comment

Column Prototype
This is a very early prototype of a persistent column. It is not expected to work yet, and leaving it open will activate other new features which will break things. Press "\" (backslash) on your keyboard to close it now.