There's a relevant discussion going on currently at https://en.wikipedia.org/wiki/Wikipedia_talk:Manual_of_Style/Mathematics starting from a proposal to shift from recommending that displayed mathematics formulas be formatted as :<math>...</math> to instead recommending <math display=block>...</math>. This proposal appears to be failing to reach consensus. The issue is that (although the display=block version generates semantically cleaner html) too many other things work less well with it, including its ability to allow deeper-than-default indentation and its ability to allow non-mathematical material such as reference footnotes on the same line.

Just a quick update from the math extension development. I added a tracking category for pages that use the deprecated mhchem syntax. It is waiting for code review at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/442124/ and currently the testing framework is broken T202223 .

Is it possible to extends this to cover the other deprecated syntax we are trying to remove \and, \bold etc.

@SalixAlba sure. However, according to my experience it's easier to get code review if the changes are not too big. So I will do that in a follow up if this one works as expected.

As mentioned here: T208301, currently there is a problem with punctuation, as it only renders the point and comma as it is used in English, but not everybody uses this system. Currently, I would say that there are more Wikipedia languages using a comma por decimal marking that a point.

@Pkra About inlining SVG: Maybe you are right and in principle SVG could become a better solution than HTML. I don't like the idea of taking the current system and just removing the image tags because this is not enough to solve those problems. The most probable case is that we don't have enough manpower to actually do the things that "might be doable" and "require a bit of work" and the result is that we are stuck with an over-customized, substandard, unmaintained system forever.

In other words: Who wants to spend months of work on developing and maintaining such a rendering-system when everyone else just puts two lines of code on his homepage and gets a much better rendering system where those problems have been largely solved years ago. Why can we not do the same, i.e. just use MathJax 3.0 with server-side pre-generation out of the box? If that renders inline SVGs, fine. If that needs special changes to work together with MediaWiki, we should try to push those changes to the original Mathjax and not create our own MathJax-clone.

I am asking because the next Community-Wishlist-Survey is coming up and I would improve and re-submit the proposal from last time
https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2017/Reading/Functional_and_beautiful_math_for_everyone

@Debenben what is currently being done is close to what you propose. With two lines of code you can't render LaTeX. With two lines you can delivfor er tex code and instruct the browser to download and run a javascript program that then renders it.
The main problem from my perspective is code review. It needs a second person feeling responsible to review the patches for Math so that there is someone that writes and another person that reviews the code. See for example the relatively simple for the tracking category for deprecated chem syntax as discussed above https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/442124/

What is currently being done is delivering the same static svg images to everyone. What I propose is to do the final rendering client-side like in normal MathJax, so that e.g.

<span style="color: red; font-family: sans; font-size: 20px; font-weight: 800">this <math>\text{whole sentence}</math> is printed in red in the browsers sans-serif font, <math>this</math> is also red with the math font on the page chosen to fit the standard text font and <math>\frac{\eqref{somelabel}}{\eqref{otherlabel}}</math> adjusts its size and content to whatever somelabel and otherlabel have been assigned before.</span>

@Debenben I think we're agreeing more than it may seem.

just use MathJax 3.0 with server-side pre-generation out of the box?
If that needs special changes to work together with MediaWiki, we should try to push those changes to the original Mathjax and not create our own MathJax-clone.

The Math Extension is already using MathJax (via the official MathJax-node APIs for server-side rendering) and I'm definitely not proposing to change that. But I would agree that the Math Extension does not use MathJax "out of the box" because that (for me) would mean inlining MathJax's SVG output (as it would be done on the client). Beyond that there are some MathJax configuration changes that could improve things (e.g., switch fonts for better Unicode coverage).

But neither of these are anything special, just typical MathJax configuration stuff. Definitely no "MathJax clone".

I don't like the idea of taking the current system and just removing the image tags because this is not enough to solve those problems.

Could you elaborate on the problems you have in mind here and why you think inlining wouldn't solve them?

I suspect that if there's a problem that's not due to the img tag hacks then it's likely a problem of MathJax itself. While it may have to do with the limits of server-side rendering, chances are that client-side MathJax rendering will have the same issue. But either way, Wikipedia would be a strong incentive for the MathJax team to find solutions for any issues and reversely WMF could fairly easily support some MathJax development to ensure that.

@Physikerwelt wrote

The main problem from my perspective is code review. It needs a second person feeling responsible to review the patches for Math so that there is someone that writes and another person that reviews the code.

I'd be interested in learning more about the code base. Are there any hackathons coming up near you? Or will you drop by the old capital some time?

@Debenben wrote

<span style="color: red; font-family: sans; font-size: 20px; font-weight: 800">this <math>\text{whole sentence}</math> is printed in red in the browsers sans-serif font

That's tricky at high quality when rendering server-side but in many cases that shouldn't be a problem; seems worth trying out to configure MathJax accordingly. It would require inlining SVGs however.

, <math>this</math> is also red with the math font on the page chosen to fit the standard text font

There's no "standard text font" on Wikipedia sites, correct? There's not even a system font stack, just font-family: sans-serif or some such, I think. So that indeed seems fairly impossible to match on the server.

Then again tweaking the font size a bit migh be sufficient for most users. The community could even gather data on how client-side MathJax sets the font-size on their systems and that data could be used to determing a suitable configuration.

@Pkra I am at Uni Wuppertal. The "old capital" is quite close. I have searched for new math contributors on several past hackathons e.g. Barcelona this year. I can introduce you to the codebase.

@Debenben anyone who want's client-side MathJax rendering (for whatever reason) can get it by enabling the source rendering mode (for text browsers) and adding the two lines of configuration you are talking about. I don't really understand what you are looking for?

anyone who want's client-side MathJax rendering (for whatever reason) can get it by enabling the source rendering mode (for text browsers) and adding the two lines of configuration you are talking about.

You don't even need to change the renderer. My answer at https://stackoverflow.com/a/31924873/1339651 still works, I think.

@Pkra great. Are we allowed to add this to the [[mw:Extension:Math]]

@Physikerwelt that uses external Javascript however. WMF doesn't allow any javascript that is not hosted from within wikimedia (for privacy reasons). The MathJax library code needs to be coming directly from the extension.

With the recent JS attacks and the CSP protection in the process of being added to counter those attacks, likely that referred to snippet will actually stop working somewhere within the next year (because it loads a library from an untrusted source).

@Pkra great. Are we allowed to add this to the [[mw:Extension:Math]]

If you are asking about licensing, stackoverflow user content is always licensed as cc-by-sa.

@TheDJ How can WMF not allow Javascript? I think users must have the control what software they execute in their browsers. I think that special small group of people that explicitly wants to use the exact method that StackOverflow and others sites use should have the opportunity to do so. The extension generates MathML and SVG, which should be good enough for the majority of STEM users.

They allow JavaScript, but there will be CSP protections that will restrict where you can load external resources from. Users aren't careful enough about what external JS they load into wikimedia users' sessions. And even if they were, not everyone's account security is up to it.

Ok. Does that mean that the following line of @Pkra post
mw.loader.load('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js?config=MML_HTMLorMML-full');//load MathJax with a suitable combined config file
might become dysfunctional in the future?

I would expect so, possibly unless T208188 goes through depending on the details.

Re there being a "special small group of people" who want the same quality of rendering that all other mathematics sites on the web have, and being willing to log in and install special Javascript to achieve it: No. What I want is for all Wikipedia readers to achieve that quality of rendering by default.

@DavidEppstein this is out of question. Even father the quality of the SVG / MathML rendering should be superior to client-side MathJax rendering. However, MathJaxs interaction and context menu functionality is something for "special users".

Do you really think the quality of SVG rendering (MathML is a joke, it doesn't even exist on most browsers) is up to the standard of client-side MathJax, let alone surpassing it? I don't.

@DavidEppstein it's not relevant what I think. A priory, I see no logical reason why the quality of server side generated SVG (by the way using MathJax) should be worse than client side generated output using the same software. For MathML one of the most cited mathematical references the DLMF (e.g., https://dlmf.nist.gov/5.2#E1) uses MathML for browsers that support it. However, I have to admit that proof by example is not a strong argument.

@Physikerwelt For example in the mobile view:

wikipedia server-side svg:

Also, stuff like <math>\text{ഗ്ദ്ധ്രീ}</math> doesn't get rendered properly with server side svg rendering. Maybe those things can be fixed, but then we are still far away from having the full functionality like line-breaking, references etc. that you get with normal mathjax and we still have accessibility issues like T194768. I can accept if you don't get those nicely wrapped equations, but something which should always work is printing the source-code. With the client-side JS approach you can be sure that you always get at least the source code, because if for some reason the JS is not loaded you see it and otherwise you can right-click and select source-code rendering in the context menu.

The misaligned mis-sized rendering of Debenben's server-side example, and the inability to properly render \text, are a big part of why so many Wikipedia mathematics editors avoid math mode altogether and use template hacks instead. (Another issue not mentioned by Debenben is that when math is included in a wikilink or external link, as it might be for instance in the title of a mathematics reference, it does not end up colored to make the link visible.) We need to aim for something that is actually uniformly at least as good as current client-side rendering results (regardless of whether it is actually client-side or server-side) in order to make math mode more widely adopted. Sticking our heads in the sand and declaring that the current svg render is already perfect (when it manifestly is not) does not advance that cause.

Sticking our heads in the sand and declaring that the current svg render is already perfect (when it manifestly is not) does not advance that cause.

Neither does saying "it must be done this way", without apparent regard to the technical requirements that the WMF is levying on you and which Physikerwelt has already made clear to you.

Did you miss the part where I said "regardless of whether it is actually client-side or server-side"? Because saying "it must be done this way" is so far from what I actually said that it appears you are responding to a straw figure of your imagination rather than to my comment.

At a minimum, a good solution must react to the choice of fonts and font sizes that are presented to the reader by the browser for text, and make compatible choices of font, font size, and alignment for the formulas. The complication is that the server cannot know the actual font that will be presented, because of the way CSS allows the browser a sequence of fallback choices for font. But this is not an insurmountable problem for server-side solutions. One could imagine doing this client-side (as most mathjax installations do), or doing a server-side construction of html code that puts the right characters in the right fonts in the right places, or doing a server-side construction of an svg file that puts the right characters in the right fonts in the right places as svg text objects (taking advantage of the fact that text in an svg file is controlled by the same css as text in html). For that matter, if MathML worked, it would also do this, but I think we must give up hope of it ever working on a widespread basis. The solution that doesn't work as well and as far as I can tell will never work as well is the one we have now in which the server makes all the choices and renders the formula as a png or svg image.

In T195861#4728232, @Physikerwelt wrote:

Ok. Does that mean that the following line of @Pkra post
mw.loader.load('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js?config=MML_HTMLorMML-full');//load MathJax with a suitable combined config file
might become dysfunctional in the future?

Yes. Most definitely.

Basically, if its something we want to support, than Math extension should but the JS on to our servers (assuming licensing is ok) and load via resource loader.

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage.

@Debenben what browsers those screenshots done with? Such a bad rendering shouldn't happen with the current solution.

Overall, it might help to pick a suitable page and build samples that apply different rendering solutions.

Again, I'd suggest to have a face-to-face meeting. This discussion continues to go in circles and I suspect some direct interaction might help. Maybe using a W3C MathOnWeb CG meeting could be helpful for providing some space for discussions with a wider input.

@Bawolff

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage

That was all that was suggested earlier. Thanks for clarifying!

In T195861#4730952, @Pkra wrote:

@Bawolff

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage

That was all that was suggested earlier. Thanks for clarifying!

Note that you cannot host fonts in mediawiki or user subpages however. So you would have to install those locally, or again fallback on integrating with resource loader.

Note that you cannot host fonts in mediawiki or user subpages however. So you would have to install those locally, or again fallback on integrating with resource loader.

Right. Of course people could start inlining the fonts as base64 in a stylesheet. As long as inline style attributes are not blocked, it should be possible to make MathJax work (we're still a bit away from equation rendering without inline styles).

Maybe wecould convince commons to allow font uploads

I think the MathJax fonts are licensed under the SIL Open Font License — at least that's what https://github.com/mathjax/MathJax/issues/1856 from a year ago implies they should be licensed under. However, that license has a no-commercial-redistribution clause (you can't sell the fonts) which may conflict with the sorts of licenses commons allows.

In T195861#4730951, @Pkra wrote:

@Debenben what browsers those screenshots done with? Such a bad rendering shouldn't happen with the current solution.

The screen-shot was taken on my home computer (debian stretch, firefox deactivated mathml, open article, scroll down to the very bottom and click on mobile view). At work I did a quick test with windows 10 / firefox, internet explorer, android tablet / samsung & other mobile browsers. Without special plug-ins or user-scripts I could not find a single device /configuration where the current rendering of the mobile view:

https://de.m.wikipedia.org/wiki/Maxwell-Gleichungen#Makroskopische_Maxwell-Gleichungen

is significantly better. I think more than 50% of readers get the mobile view, so this example shows what it currently looks like for the majority of readers.

In T195861#4730635, @Izno wrote:

Neither does saying "it must be done this way", without apparent regard to the technical requirements that the WMF is levying on you and which Physikerwelt has already made clear to you.

What do you mean by "technical requirements that the WMF is levying on you?". I am not aware of any technical requirements other than the JS requirements @TheDJ mentioned above where I cannot see a technical blocking point. Nobody from WMF has come forward with any requirements so far, I think @Bawolff is the first one actively participating. Restrictions for user-scripts are irrelevant because the solution must be something that everybody gets by default. If everybody viewing a page would be downloading the fonts from commons you could immediately break the math rendering on all projects with accidental changes to the font-files and I am not sure how the commons servers would handle the additional load.

The screen-shot was taken on my home computer (debian stretch, firefox deactivated mathml, open article, scroll down to the very bottom and click on mobile view). At work I did a quick test with windows 10 / firefox, internet explorer, android tablet / samsung & other mobile browsers. Without special plug-ins or user-scripts I could not find a single device /configuration where the current rendering of the mobile view:
https://de.m.wikipedia.org/wiki/Maxwell-Gleichungen#Makroskopische_Maxwell-Gleichungen

The mobile view simply has a bug here. I've filed https://phabricator.wikimedia.org/T209148

Here's a random idea for interested people to try.

The following snippet

document.querySelectorAll('.mwe-math-fallback-image-inline').forEach(
    (img) => {
fetch(img.src)
    .then(r => r.text())
    .then(text => {
        img.outerHTML = text;
    })
    .catch(console.error.bind(console));
    }
)

replaces the equation img tags with the relevant svg source (note: because I'm lazy I only wrote this for modern browsers).

If anyone gives this a spin, I'd be interested in hearing how it affects rendering. For example, equations should inherit color and text elements in the svg should render better (slightly; it's still using monospace).

I was looking today for how I am supposed to format commutative diagrams with the current Wikipedia mathematics support. https://en.wikipedia.org/wiki/Help:Displaying_a_formula#Commutative_diagrams says we should just give up, make our own image files offline, and then upload them to commons. But obviously that's going to be even worse for matching fonts and font sizes than the current Wikipedia mathematics support.
This is something MathJax can support: https://math.meta.stackexchange.com/questions/2324/how-to-draw-a-commutative-diagram
I think we should include our non-support of this functionality (assuming my inability to find documentation means it really is non-supported) as a serious deficiency of our current level of mathematics formatting, and add it to the list of things we need to do better.

Hi everyone,

Volker Sorge (@Zorkow) and I (@Pkra) have drafted a grant proposal for 2019 at [1] titled "Accessibility of equation rendering: a comparative evaluation".

The gist of it from our summary:

We propose a study to analyze the accessibility of equation rendering techniques on the web. This evaluation will focus in particular on the techniques currently used on Wikimedia sites as well as alternatives that are feasible to adopt for Wikimedia.

It would be very helpful to get your feeback on this proposal!

If you could help spread this further in the community (or have recommendations for us on doing that), that would be very useful as well. And of course if you think you can endorse this proposal, that would be extremely helpful.

Thanks,
Peter

[1] https://meta.wikimedia.org/wiki/Grants:Project/Accessibility_of_equation_rendering:_a_comparative_evaluation

@DavidEppstein wrote

I think we should include our non-support of this functionality (assuming my inability to find documentation means it really is non-supported) as a serious deficiency of our current level of mathematics formatting, and add it to the list of things we need to do better.

There's https://phabricator.wikimedia.org/T30258 where @Physikerwelt suggested to add AMScd. (Though perhaps the specific nature of AMScd notation might prove an obstacle for texvc.)

To come back to the original intent of this ticket. I think we now have a committee. Maybe we can make it a little bit more formal, to have better chances to actually change things. I drafted a community user page here https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math. Otherwise I would say the "inofficial" commitee is:

@Debenben
@Physikerwelt
@Andreg-p
@SalixAlba
@Pkra
@mhchem
@DavidEppstein
@KaiMartin
@Framawiki
@MGChecker
@WhitePhosphorus
@mobrovac
@Mathmensch

I don't think I have knowledge and time to really contributed, I just subscribed this task because I'm interested in development and improvement of the Math extension.

@Physikerwelt I'm in.

@DavidEppstein regarding commutative diagrams some stray thoughts.

I came across T56213 the other day which is tracking issues related to "Infographics, graph, chart, diagram, plotting, animation support". Since many of the tracked issues appear stuck, it might be an opportunity for communities to work together.

That may, however, require giving up the idea of having specific solutions (i.e., xy, pstricks or tikz).

For example, SVG is the natural medium for diagrammatic content on the web. T40271 suggests adding a general purpose SVG editor that provides raw SVG editing as well. It's not too hard to write SVG (especially in comparison to tikz etc) so perhaps editors might be ok with manually authoring SVGs (if they can pull in <math> content)?

There are also linear syntax options with tooling that might be easier to vet by WMF later on (e.g., DOT, JessieCode) and those might be useful more generally both in terms of the wider community as well as in technical terms (e.g., accessibility).

@ all is there community support for T32215?

To be accepted as a user group the following is neccary:

Thank you for your recent application to obtain Wikimedia User Group status and your plans to advance Wikimedia Community User Group Math. Below is the list of criteria you must meet before we can move onto full AffCom review. Any criteria not yet met are explained below:

You have met 4 out of 8 criteria:

At least 3 active Wikimedia editors: CRITERION NOT YET MET
Focus advances Wikimedia: CRITERION MET
Mission aligned with Wikimedia Foundation: CRITERION MET
Compliance with naming guidelines and trademark policy: CRITERION MET
Information about group published on a Wikimedia wiki: CRITERION MET
Plans for activities or efforts to advance Wikimedia projects: CRITERION NOT YET MET
Allows new members: CRITERION NOT YET MET
Two designated contacts for Wikimedia Foundation: CRITERION NOT YET MET

At least 3 active wikimedians
We have noticed that you currently do not meet the minimum requirement of having at least 3 active wikimedians as part of your user group. An active wikimedians for the purposes of a user group application is a user who has at least 300 edits and has had a user account for at least 6 months. Are there other users who are part of your user group who you could add to the user group page and meet these requirements?

Minimum Numbers (Recommendation only):
I do not see any editors listed as interested in the group currently. I imagine you are working to recruit more editors to become members and know that can change quickly. The recommended number is at least 10 members to apply to be a user group. While this is not a requirement, I wanted to check, do you have plans for recruit additional members?

Logo (Notification only):
I do not yet see a logo on your page, while it is not required that you have a logo to be recognized as a user group, please see the logo best practices page to learn which logos you may use for your user group (see: https://meta.wikimedia.org/wiki/Wikimedia_movement_affiliates/Logos_best_practices#wugs)

Planned Activities:
We currently do not see any planned activities with specific dates, such as meetings, events, edit-a-thons, etc. on your user group page. Do you have any planned events in the next few months? If so, please add them to the user group page.

Allow New Members:
It is unclear from your application and user group page how you will allow new members to join. Please be sure to add a note as to how other members may join to your page.

Key Contacts:
Two designated contacts for Wikimedia Foundation - The Affiliations Committee and the Wikimedia Foundation staff from time to time need to get in touch with primary contacts from affiliates for various reasons. Can you please specify which two are the primary contacts for your group?

Once we resolve these points I can advance your application for full committee review.

@SalixAlba We have to change our algorithms to look for math tags. dewikiversity is using {{#tag: math}} via https://de.wikiversity.org/wiki/Vorlage:Math on more than 18 000 pages.

Is someone interested in moving the User Group application forward. I think https://meta.wikimedia.org/wiki/Wikimaps_User_Group is an example for a comparable active usergroup. @Thuvack reminded me a few days ago to complete the draft on https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math but I won't find time for this in the near future.

@Physikerwelt please remove me from the list above.

Some things I wanted to mention which might help the visibility of this post since I think it is a pretty important issue:

It took me around 3 hours to find how TeX input is currently actually managed. Actually I wanted to find a formal specification of the actual accepted subset of LaTeX, not just "some parts of AMSTeX and color and some odds and ends you'll have to find out for yourself". The pages I consulted were:
https://meta.wikimedia.org/wiki/Texvc
https://meta.wikimedia.org/wiki/TeX
https://www.mediawiki.org/wiki/Manual:Enable_TeX

The content of those pages led me to believe that the old tex OCaml plugin was what was still in use, which I then tried to find the source code of.
https://en.wikipedia.org/wiki/User:Taw
https://github.com/dlitz/texvc
https://phabricator.wikimedia.org/diffusion/EMAT/browse/master/modules/ve-math/tools/

Then I finally found the mathoid git repo.

https://github.com/wikimedia/mathoid

So I think it would be helpful if the Meta pages mentioned above were corrected to reflect the current stand of things so people don't have to go on internet odysseys to find this stuff out.

@GregorAlexandru that's an excellent idea. I think the following resources are helpful.

Do you think it would be a good idea to list the allowed commands

https://github.com/wikimedia/texvcjs/blob/master/lib/texutil.js

in the texvc package documentation? That's something I could easily do.

https://mirror.informatik.hs-fulda.de/tex-archive/macros/latex/contrib/texvc/texvc.pdf

I did not do it in the first place, because I thought it might be too much.

@Physikerwelt
I don't think they belong in the package documentation of the latex package "texvc" since that contains only the macros newly defined, not the subset of LaTeX supported by the validator-converter, texvcjs.
I think it belongs in the documentation of texvcjs which should be lnked to (at least) from the https://meta.wikimedia.org/wiki/Help:Displaying_a_formula article.

Especially the comments "[…] a subset of TeX markup, including some extensions from LaTeX and AMS-LaTeX"
and "Thus, only a limited part of the full TeX language is supported; see below for details"

are frustrating, if you're looking for a specific answer.
https://github.com/wikimedia/texvcjs/blob/master/lib/texutil.js
is probably a good enough documentation since it contains all the information, though probably not directly understandable to a non-programmer.

@GregorAlexandru We are actually planning to get rid of the texvcjs so that in the future we can just refer to the MathJax documentation.

Resolving! With https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math , there is a vehicle to separate decision making from the actual implementation.