Page MenuHomePhabricator

Implement native MathML rendering in Math
Open, MediumPublic

Description

Currently, the Math extension is quite complex and comes with dependencies to

  • mathjax
  • mathoid
  • texvcjs
  • texvcinfo
  • restbase
  • (database) tables and cache

It converts a string given in a relatively simple LaTeX dialect to MathML and then to SVG PNG. The new MathML standard and the announced implementation of MathML in Chrome and Edge native MathML rendering will be sufficient for most users. The remaining user group (client-side MathJax) can serve as a bridge technology until all Browsers support MathML.

After a brief discussion in the W3C Math WG, the implementation of a native PHP to MathML converter seems feasible and doable. While the codebase of the Math extension might grow, the complexity of the setup decreases significantly.

The most incremental way is to implement the native MathML rendering in texvcjs and then port it 1-1 to PHP using phpeggy.
As a preparation step, one might also want to merge texvcjs and texvcinfo.

References:

  • Stackoverflow has some hints on realization of LaTeX to MathML in php.

Related Objects

StatusSubtypeAssignedTask
StalledStegmujo
OpenNone
OpenNone
OpenPhysikerwelt
ResolvedStegmujo
Resolvedmmartorana
ResolvedPhysikerwelt
ResolvedStegmujo
DuplicateStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
DuplicateNone
DeclinedNone
OpenNone
DeclinedNone
OpenStegmujo
StalledNone
ResolvedNone
ResolvedStegmujo
ResolvedPhysikerwelt
ResolvedStegmujo
ResolvedPhysikerwelt
ResolvedStegmujo
ResolvedBUG REPORTStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
DuplicateStegmujo
DuplicateStegmujo
DeclinedStegmujo
DeclinedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedNone
ResolvedStegmujo
ResolvedStegmujo
ResolvedPhysikerwelt
ResolvedPhysikerwelt
DuplicateNone
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
ResolvedStegmujo
OpenStegmujo
OpenStegmujo
ResolvedStegmujo
ResolvedBUG REPORTStegmujo
ResolvedBUG REPORTStegmujo
OpenBUG REPORTNone
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedBUG REPORTPhysikerwelt
ResolvedPhysikerwelt
ResolvedPhysikerwelt
ResolvedPhysikerwelt
ResolvedBUG REPORTStegmujo
ResolvedStegmujo
InvalidNone

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Physikerwelt set Due Date to Oct 31 2022, 11:00 PM.Jun 9 2022, 7:09 AM
Physikerwelt updated the task description. (Show Details)

@Physikerwelt The nice thing about our Math setup now is that it works without JavaScript. If a user does not have a browser which supports MathML and they disable JavaScript, client-side MathJax isn't going to work. I imagine some degree of consultation would be necessary if we are to remove maths formulae from non-JS users.

@dom_walden yes. Certainly, this was a big advantage in the past. However, as Chrome will support MathML in the near future. The number of users without JS and without MathML support will be very small. Moreover, as you might have already realized the support for PNG rendering is based on outdated technology and would need to be redone.
In general, I agree that we should reach out for larger community involvement (not only the Math mailing list but maybe also wikitech-l?). However, I guess we should do a few more preliminary steps before we go on with a wide announcement/community consultation.

Change 826353 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/mathoid@master] Replace texvcinfo with mathoid-texvcjs

https://gerrit.wikimedia.org/r/826353

Change 826353 merged by jenkins-bot:

[mediawiki/services/mathoid@master] Replace texvcinfo with mathoid-texvcjs

https://gerrit.wikimedia.org/r/826353

Change 869796 had a related patch set uploaded (by Stegmujo; author: Stegmujo):

[mediawiki/extensions/Math@master] Add length constants for MML generation

https://gerrit.wikimedia.org/r/869796

Change 869796 abandoned by Stegmujo:

[mediawiki/extensions/Math@master] Add length constants for MML generation

Reason:

This will be reupluaded in another changeset

https://gerrit.wikimedia.org/r/869796

Change 871131 had a related patch set uploaded (by Stegmujo; author: Stegmujo):

[mediawiki/extensions/Math@master] Add updated TexConstants and Lengths

https://gerrit.wikimedia.org/r/871131

Change 871131 merged by jenkins-bot:

[mediawiki/extensions/Math@master] Add updated TexConstants and Lengths

https://gerrit.wikimedia.org/r/871131

Change 871200 had a related patch set uploaded (by Stegmujo; author: Stegmujo):

[mediawiki/extensions/Math@master] Add test stub for basic MML Methods

https://gerrit.wikimedia.org/r/871200

Change 871200 merged by jenkins-bot:

[mediawiki/extensions/Math@master] Add test for basic MML Methods

https://gerrit.wikimedia.org/r/871200

Change 871278 had a related patch set uploaded (by Stegmujo; author: Stegmujo):

[mediawiki/extensions/Math@master] Add tests for more TeX statements

https://gerrit.wikimedia.org/r/871278

Change 871278 merged by jenkins-bot:

[mediawiki/extensions/Math@master] Add tests for more TeX statements

https://gerrit.wikimedia.org/r/871278

@Physikerwelt: Hi, the Due Date set for this open task passed a while ago.
Could you please either update or reset the Due Date (by clicking Edit Task), or set the status of this task to resolved in case this task is done? Thanks!

The native mode can now be enabled on beta. Create an account and select MathML rendering under preferences.

Screenshot 2023-09-18 at 10-58-34 Preferences - Wikipedia the free encyclopedia.png (261×804 px, 36 KB)

This is now also live on testwiki. @Jdlrobson do we need your approval to enable the rendering option on all wikis, or is approval from the wikimedia-community-group-math sufficient?

@Physikerwelt what does enabling this rendering on all wikis mean from a performance perspective? Has there been any prior analysis of performance implications? Does it load additional CSS/JavaScript?

We recently added performance budgeting to Vector 2022, so one quick thing you can do is submit a Gerrit patch changing the configuration flag to the new default and see if it trips any of our performance tests.

I've created https://test.wikipedia.org/wiki/Help:Displaying_a_formula with a copy of the English Wikipedia version. You need to switch your preferences to MathML to see it.

Most things seem fine, although with a lighter weight, and font size for equations a little less than standard font for surronding text. A couple of things are broken
\tbinom{n}{k}

\begin{align}
f(x) & = (a+b)^2 \\
& = a^2+2ab+b^2 \\
\end{align}

Has not aligned correctly. It can be made to perform a bit better, if a second ampersand is added

\begin{align}
f(x) & = & (a+b)^2 \\
& = & a^2+2ab+b^2
\end{align}

But chrome is aligning each part in the centre of its space rather than left aligning. It also not respecting alignment info in \begin{array}{lcr}.

There is not much distinction in bracket sizes ( \bigl( \Bigl( \biggl( \Biggl( etc.

Some of <chem> environment is broken <chem>A ->[{}\atop\ce{+H2O}] B</chem> with a Failed to parse syntax error, which is a bit weird, as other chem stuff works.

I'm not sure where the issues lie, is it the conversion of latex to mathml, or Chrome's rendering of the mathml.

@Physikerwelt what does enabling this rendering on all wikis mean from a performance perspective? Has there been any prior analysis of performance implications? Does it load additional CSS/JavaScript?

It pushes compute power from the mathoid instances to the MediaWiki worker nodes. The effect on the HTML output is that it removes links to SVG images and unhides MathML elements. By design, the MathML elements are identical to the current MathML elements. I did run a quick environmental impact test on digitalbeacon.co. It computed a Co-2 footprint of 0.181g for the new rendering compared to the old rendering with 0.771g. This is a significant improvement. We used the following pages

https://en.wikipedia.beta.wmflabs.org/wiki/Help:MathTestNative (new)

https://en.wikipedia.beta.wmflabs.org/wiki/Help:MathTest (old)

It currently does not load additional CSS and JS, but depending on the user feedback, a font fallback mechanism might be needed, cf. T348790.

We recently added performance budgeting to Vector 2022, so one quick thing you can do is submit a Gerrit patch changing the configuration flag to the new default and see if it trips any of our performance tests.

I am not entirely sure how to do that. First, we must add $wgMathValidModes[] = 'native'; since the new mode is disabled by default. Second, we must set $wgDefaultUserOptions['math'] = 'native'; . However, I don't know what settings are used by the test you are referring to.

I've created https://test.wikipedia.org/wiki/Help:Displaying_a_formula with a copy of the English Wikipedia version. You need to switch your preferences to MathML to see it.

Most things seem fine, although with a lighter weight, and font size for equations a little less than standard font for surronding text. A couple of things are broken
\tbinom{n}{k}

\begin{align}
f(x) & = (a+b)^2 \\
& = a^2+2ab+b^2 \\
\end{align}

Has not aligned correctly. It can be made to perform a bit better, if a second ampersand is added

\begin{align}
f(x) & = & (a+b)^2 \\
& = & a^2+2ab+b^2
\end{align}

But chrome is aligning each part in the centre of its space rather than left aligning. It also not respecting alignment info in \begin{array}{lcr}.

There is not much distinction in bracket sizes ( \bigl( \Bigl( \biggl( \Biggl( etc.

Some of <chem> environment is broken <chem>A ->[{}\atop\ce{+H2O}] B</chem> with a Failed to parse syntax error, which is a bit weird, as other chem stuff works.

I'm not sure where the issues lie, is it the conversion of latex to mathml, or Chrome's rendering of the mathml.

@SalixAlba, thank you. I will create individual tickets for the problems.

@Physikerwelt I don't see anything wrong with deployment at a glance, but if you could deploy this on a few pilot wikis first (for about 1-2weeks) before enabling everywhere we can monitor performance dashboards here: https://grafana.wikimedia.org/d/000000232/navigation-timing-by-country?orgId=1&search=open&query=folder:current

You know we have our own PEG.js fork with PHP generation called WikiPEG.

You know we have our own PEG.js fork with PHP generation called WikiPEG.

I was not aware that it could generate PHP code. It would be interesting to see if current grammar can also be compiled with wikiPEG and to compare the performance to phpeggy.