Page MenuHomePhabricator

Create a committee to improve the math support in Wikimedia projects
Open, LowPublicDec 30 2018

Description

Until Thursday, May 24, 2018, the syntax of the input for mathematical formulae was fixed. Now, we are in a position that we can change the input syntax. Before doing so, we should agree on (realistic) goals how to change the syntax.

Concerning the input syntax it has been suggested to

  • remove all modifications done by texvcjs
  • support unicode based input
  • add (more) semantic macros
  • support referencing (\label, \tag, \eqref...)
  • support wikilinks (\href)
  • create a viable distinction of inline and block formulae
  • rethink <math> <math chem> and <chem> redundancy
  • ...

    see https://www.mediawiki.org/wiki/Special:MyLanguage/Extension:Math/Roadmap for the current proposed update strategy.

Concerning the output it has been suggested to

  • get a WMF staff member to join the commission and outline technical requirements
  • use JS and webfonts
  • match math and non-math text as good as or better than w:en:Template:Math for all devices
  • improve rendering to be as good as or better than client-side MathJax on math.stackexchange.com for all devices
  • ...

We would like to develop a consistent proposal and describe it in a way that can be understood by the different groups affected. That is

  • the average Wikipedia contributor,
  • advanced users with experience in TeX
  • and software developers.

Please add your name to the list if you are interested in contributing

Details

Due Date
Dec 30 2018, 11:00 PM

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes

@all: Concerning the output:

I think we should suggest the common-html output of mathjax v3 with server side pre-processing as a default. It should be faster and better than the pure client-side mathjax solution of the past.

One reason why we don't get feedback concerning JS and webfonts might be that people don't understand what this would improve and they don't understand how it would affect them because it is too abstract. I would suggest to create a "math2020wiki" website on toolforge like the "articleplaceholderwiki", install MediaWiki out of the box without math extension and install mathjax v3 out of the box and with a little configuration we have a wiki for live demonstration, at least concerning the user interaction and we can point to some editors / extensions / gadgets that would fail.

Then we force everyone who should be concerned about it to give us feedback before putting further work into it.

@TheDJ Since you were involved in the client-side mathjax project of the past, it would be nice if you could help us to avoid problems that killed it last time.

Pkra added a comment.EditedJun 22 2018, 8:09 PM

I think we should suggest the common-html output of mathjax v3 with server side pre-processing as a default. It should be faster and better than the pure client-side mathjax solution of the past.

IIRC, back when MathJax's SVG output was chosen, the v2 CommonHTML output was considered and rejected. I think that was in part because the CommonHTML output was fairly new back then, but the technical arguments have not really changed with v3 (and performance is not slower than with SVG).

I'd be very interested in hearing more about the issues people have with the SVG output; in my experience it can reach an quality equivalent to the CommonHTML (if done right), even if both formats have trade offs. For example, fixing the SVG stroke size would be a good idea to fix printing.

Given the community's support of the templating sytem, I've been wondering whether the PreviewHTML may be worth considering. Its layout does not aim for TeX-quality layout and it currently misses a few features but it might be sufficient for existing content and is very simple code to improve (also, it needs no webfonts).

I'd be very interested in hearing more about the issues people have with the SVG output; in my experience it can reach an quality equivalent to the CommonHTML (if done right), even if both formats have trade offs. For example, fixing the SVG stroke size would be a good idea to fix printing.

The SVG output still does not match the body text in font size, font weight, and baseline (although it is a lot better than it used to be). This becomes particularly apparent when using roman fonts within math, for instance via operatorname. Unlike html, svg-formatted math stays black when linked or wikilinked, making links that include formulas problematic. One cannot select SVG-output math formulas to highlight them, nor (because of this highlighting issue) is it obvious that one can copy and paste from them. When one does copy and paste, the result is the TeX source rather than what one expects (maybe this is ok?) but the pasted text also includes extraneous markup like \displaystyle and braces that were not part of the original markup.

Given the community's support of the templating sytem, I've been wondering whether the PreviewHTML may be worth considering. Its layout does not aim for TeX-quality layout and it currently misses a few features but it might be sufficient for existing content and is very simple code to improve (also, it needs no webfonts).

My impression is that the community support of the templating system is almost entirely based on the perception that the rendering of math formulas using SVG output is still not up to par.

@Pkra DavidEppstein already said the most important issues:

  • Properly render things like <math>\text{это хорошо выглядит?}</math>, <math>\text{für alle}</math> to have the same quality and appearance of the text outside
  • The ability to copy-paste parts of an equation

Some other things I would add to the list:

  • Linebreaking for small screens, snippet-previews etc. (some people put linebreaks like <math>a</math>{{nowrap|<math>\;=b</math>.}} )
  • A referencing system with automatic numbers aligned left or right

Except for the copy-paste problem I see them as blocking issues which we need to have solved before we can address the "create a viable distinction of inline and block formulae" problem, because otherwise you need the capability of composing one block-formula with several math tags and templates and the :<math> markup which in turn produces invalid html, some more rendering issues and also problems for unexperienced editors that try to edit formulas with the VE.

If those problems cannot be solved with SVG, we need something else. I am not familiar enough with preview html to make a judgment on that option, but I also don't see a reason against common html, so why aim for a degraded rendering system if almost every other website has a better one?

TheDJ added a comment.EditedJun 23 2018, 7:45 AM

@TheDJ Since you were involved in the client-side mathjax project of the past, it would be nice if you could help us to avoid problems that killed it last time.

I'll give you the list of some of the requirements for javascript in MediaWiki that would apply here:

  • Has to have a non-JS fallback
  • JS should only be included on the pages that make use of Math
  • Should be triggered by mw.hook('wikipage.content') instead of document ready
  • Should lazy load and load asynchronous
  • Should process the elements in the page sequential and with interrupts (so the page isn't blocked for too long on math processing)
  • Should be accessible and at the least not cause additional accessibility problems
  • Should not reflow the page (resize elements during page load and make stuff "jump around")
  • Should pass a review by security team
  • Webfonts should not cause 'blank text' while the page loads (there is a css option for this now, but not sure how well it performs).
  • Overall payload should not be so huge that it might cause problems for people on low speed connections or low power (old) devices.

@TheDJ Thanks for the requirement list. Am I right that those are user-convenience features we "should" meet and not "hard" requirements breaking the MediaWiki software? I have the impression that lazy and asynchronous loading might conflict with the requirement to not reflow the page.

but the technical arguments have not really changed with v3

@Pkra Can you explain what they were? I was not involved in the discussion and did not have much background knowledge about math rendering on websites at the time, so the impression I got / the main argument that convinced me as a Firefox user was that SVG would (eventually) only be a fallback for old browsers without MathML support. Also, can you say how good the JS-requirements can be met? Otherwise we have to find out by experimenting with something like a "math2020wiki" anyway.

TheDJ added a comment.EditedJun 24 2018, 11:10 AM

@Debenben No they are hard requirements for all new software deployed to Wikimedia properties, and we are continuously fixing and reworking the last of the existing code that doesn't comply with it yet. JS causing reflowing is the latest focus of the now 6 year effort to improve performance of MediaWiki and Wikipedia.

I have the impression that lazy and asynchronous loading might conflict with the requirement to not reflow the page.

No not necessarily. Not if you deliver the right CSS before the JS loads. But with image fallbacks and webfonts this is an almost impossible requirement to fulfill for javascript rendered math (because one has a fixed size, where the other is browser dependent). It should be noted however that the "not reflowing" has an addendum. For instance CentralNotice violates it (because there is no way to do it without conditional JS for unregistered users, without breaking caching) and there is still some old and even at times new code that violates this rule. Still, we are fixing code that reflows the page whenever we can and sometimes even removing features because they don't do this and either cannot be fixed or are not useful enough to invest time in.

Mathmensch updated the task description. (Show Details)Jun 24 2018, 11:24 AM

Hello,

I think it would be valuable if there was a place where mathematical editors, such as myself, could suggest macros, so that they are then implemented by the programmers capable to do it.

This would require that it is sufficiently simple for programmers to add them so that they would actually do it. (Although then probably the mathematicians could do it themselves, so that an easy-to-use macro-adding system may be the ultimate goal, as I've suggested in the past.)

For instance, I always have to write \operatorname{Aut} instead of \Aut, and the middle command is not working.

Pkra added a comment.Jun 25 2018, 8:02 AM

Just some rapid responses.

DavidEppstein wrote:

The SVG output still does not match the body text in font size, font weight, and baseline (although it is a lot better than it used to be).

It would be useful to have a careful analysis of this. IIRC mathoid does some ex-to-em conversion which may be the culprit for font size and baseline issues. I'm also wondering if it is a browser-specific problem

As for font weight, I suspect the bad stroke width configuration I mentioned might help. I could imagine there's a quality gap in older browser or across different OSs.

If it comes down to font matching of text and equation layout, it will become very difficult since (iiuc) Wikipedia does not specify anything beyond sans-serif for text (which is hard to match aganist).

This becomes particularly apparent when using roman fonts within math, for instance via operatorname.

I suspect that could be resolved by inlining SVG.

Unlike html, svg-formatted math stays black when linked or wikilinked, making links that include formulas problematic.

That would be resolved by inlining SVG.

One cannot select SVG-output math formulas to highlight them, nor (because of this highlighting issue) is it obvious that one can copy and paste from them.

That's a browser limitation of SVG. I've actually raised this with SVG WG members before and it seems they've never thought about it. This probably has something to do with the low support for copy&pasting SVG in general but it's a larger issue that would certainly benefit from support by a use case from Wikipedia.

As it happens, MathJax v3 CommonHTML output also will not be highlighted well since all text content ends up in pseudo elements.

When one does copy and paste, the result is the TeX source rather than what one expects (maybe this is ok?) but the pasted text also includes extraneous markup like \displaystyle and braces that were not part of the original markup.

IIUC that's because of texvc's sanitization; I suspect it needs to be dealt on that level.

My impression is that the community support of the templating system is almost entirely based on the perception that the rendering of math formulas using SVG output is still not up to par.

Thank you for sharing your impression.

Debenben wrote in addition

Properly render things like <math>\text{это хорошо выглядит?}</math>, <math>\text{für alle}</math> to have the same quality and appearance of the text outside

I suspect that could be solved by inlining the SVG.

The ability to copy-paste parts of an equation

While I find this a very intersting problem, I don't think there's a viable equation rendering solution for web today that could provide this.

Copy&paste for web content is fairly limited in general so I admit I find it difficult to consider that a point against any particular equation layout solution.

Linebreaking for small screens, snippet-previews etc. (some people put linebreaks like <math>a</math>{{nowrap|<math>\;=b</math>.}} )

That's doable but requires a bit of work. Again, inlining SVGs would make it easier.

But generally speaking, line-breaking is a hard problem and while MathJax does have good support for automated line-breaks, there are natural limitations (e.g., anything tabular). Especially a dynamic community like Wikipedia would probably benefit from (and lead by) good authoring practices that augment the general platform. This would help with reflow as well as accessibility.

A referencing system with automatic numbers aligned left or right

That's fairly trivial with MathJax but seems problematic for the Math Extension since equations are stored without context of the page and re-used across all instances. In my personal experience, labels are best handled outside of equation layout.

If those problems cannot be solved with SVG, we need something else.

I'd caution against drawing red lines. Some problems may not be solvable or not solvable at this time; they probably should not disqualify any particular approach or improvements.

Reversely, technical limitation should not make an impossible but important problem less relevant for a long term roadmap.

why aim for a degraded rendering system if almost every other website has a better one?

One man's trash is another man's treasure. The PreviewHTML has some properties that can be very appealing and the other outputs have their own issues. It will remain a question of trade offs for quite some time.

Vvjjkkii renamed this task from Create a committee to improve the math support in Wikimedia projects to q2baaaaaaa.Jul 1 2018, 1:07 AM
Vvjjkkii raised the priority of this task from Low to High.
Vvjjkkii updated the task description. (Show Details)
Vvjjkkii removed subscribers: gerritbot, Aklapper.
WhitePhosphorus renamed this task from q2baaaaaaa to Create a committee to improve the math support in Wikimedia projects.Jul 1 2018, 2:28 AM
WhitePhosphorus lowered the priority of this task from High to Low.
WhitePhosphorus updated the task description. (Show Details)
WhitePhosphorus added subscribers: Aklapper, gerritbot.
DavidEppstein added a comment.EditedJul 2 2018, 5:28 PM

There's a relevant discussion going on currently at https://en.wikipedia.org/wiki/Wikipedia_talk:Manual_of_Style/Mathematics starting from a proposal to shift from recommending that displayed mathematics formulas be formatted as :<math>...</math> to instead recommending <math display=block>...</math>. This proposal appears to be failing to reach consensus. The issue is that (although the display=block version generates semantically cleaner html) too many other things work less well with it, including its ability to allow deeper-than-default indentation and its ability to allow non-mathematical material such as reference footnotes on the same line.

Ebe123 added a subscriber: Ebe123.Aug 5 2018, 4:05 PM

Just a quick update from the math extension development. I added a tracking category for pages that use the deprecated mhchem syntax. It is waiting for code review at https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/442124/ and currently the testing framework is broken T202223 .

Is it possible to extends this to cover the other deprecated syntax we are trying to remove \and, \bold etc.

Aschmidt removed a subscriber: Aschmidt.Aug 19 2018, 6:01 PM

@SalixAlba sure. However, according to my experience it's easier to get code review if the changes are not too big. So I will do that in a follow up if this one works as expected.

As mentioned here: T208301, currently there is a problem with punctuation, as it only renders the point and comma as it is used in English, but not everybody uses this system. Currently, I would say that there are more Wikipedia languages using a comma por decimal marking that a point.

@Pkra About inlining SVG: Maybe you are right and in principle SVG could become a better solution than HTML. I don't like the idea of taking the current system and just removing the image tags because this is not enough to solve those problems. The most probable case is that we don't have enough manpower to actually do the things that "might be doable" and "require a bit of work" and the result is that we are stuck with an over-customized, substandard, unmaintained system forever.

In other words: Who wants to spend months of work on developing and maintaining such a rendering-system when everyone else just puts two lines of code on his homepage and gets a much better rendering system where those problems have been largely solved years ago. Why can we not do the same, i.e. just use MathJax 3.0 with server-side pre-generation out of the box? If that renders inline SVGs, fine. If that needs special changes to work together with MediaWiki, we should try to push those changes to the original Mathjax and not create our own MathJax-clone.

I am asking because the next Community-Wishlist-Survey is coming up and I would improve and re-submit the proposal from last time
https://meta.wikimedia.org/wiki/Community_Wishlist_Survey_2017/Reading/Functional_and_beautiful_math_for_everyone

@Debenben what is currently being done is close to what you propose. With two lines of code you can't render LaTeX. With two lines you can delivfor er tex code and instruct the browser to download and run a javascript program that then renders it.
The main problem from my perspective is code review. It needs a second person feeling responsible to review the patches for Math so that there is someone that writes and another person that reviews the code. See for example the relatively simple for the tracking category for deprecated chem syntax as discussed above https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/Math/+/442124/

What is currently being done is delivering the same static svg images to everyone. What I propose is to do the final rendering client-side like in normal MathJax, so that e.g.

<span style="color: red; font-family: sans; font-size: 20px; font-weight: 800">this <math>\text{whole sentence}</math> is printed in red in the browsers sans-serif font, <math>this</math> is also red with the math font on the page chosen to fit the standard text font and <math>\frac{\eqref{somelabel}}{\eqref{otherlabel}}</math> adjusts its size and content to whatever somelabel and otherlabel have been assigned before.</span>

Pkra added a comment.Nov 2 2018, 8:49 PM

@Debenben I think we're agreeing more than it may seem.

just use MathJax 3.0 with server-side pre-generation out of the box?
If that needs special changes to work together with MediaWiki, we should try to push those changes to the original Mathjax and not create our own MathJax-clone.

The Math Extension is already using MathJax (via the official MathJax-node APIs for server-side rendering) and I'm definitely not proposing to change that. But I would agree that the Math Extension does not use MathJax "out of the box" because that (for me) would mean inlining MathJax's SVG output (as it would be done on the client). Beyond that there are some MathJax configuration changes that could improve things (e.g., switch fonts for better Unicode coverage).

But neither of these are anything special, just typical MathJax configuration stuff. Definitely no "MathJax clone".

I don't like the idea of taking the current system and just removing the image tags because this is not enough to solve those problems.

Could you elaborate on the problems you have in mind here and why you think inlining wouldn't solve them?

I suspect that if there's a problem that's not due to the img tag hacks then it's likely a problem of MathJax itself. While it may have to do with the limits of server-side rendering, chances are that client-side MathJax rendering will have the same issue. But either way, Wikipedia would be a strong incentive for the MathJax team to find solutions for any issues and reversely WMF could fairly easily support some MathJax development to ensure that.

Pkra added a comment.Nov 2 2018, 8:51 PM

@Physikerwelt wrote

The main problem from my perspective is code review. It needs a second person feeling responsible to review the patches for Math so that there is someone that writes and another person that reviews the code.

I'd be interested in learning more about the code base. Are there any hackathons coming up near you? Or will you drop by the old capital some time?

Pkra added a comment.EditedNov 2 2018, 9:09 PM

@Debenben wrote

<span style="color: red; font-family: sans; font-size: 20px; font-weight: 800">this <math>\text{whole sentence}</math> is printed in red in the browsers sans-serif font

That's tricky at high quality when rendering server-side but in many cases that shouldn't be a problem; seems worth trying out to configure MathJax accordingly. It would require inlining SVGs however.

, <math>this</math> is also red with the math font on the page chosen to fit the standard text font

There's no "standard text font" on Wikipedia sites, correct? There's not even a system font stack, just font-family: sans-serif or some such, I think. So that indeed seems fairly impossible to match on the server.

Then again tweaking the font size a bit migh be sufficient for most users. The community could even gather data on how client-side MathJax sets the font-size on their systems and that data could be used to determing a suitable configuration.

Theklan removed a subscriber: Theklan.Nov 5 2018, 10:40 AM

@Pkra I am at Uni Wuppertal. The "old capital" is quite close. I have searched for new math contributors on several past hackathons e.g. Barcelona this year. I can introduce you to the codebase.

@Debenben anyone who want's client-side MathJax rendering (for whatever reason) can get it by enabling the source rendering mode (for text browsers) and adding the two lines of configuration you are talking about. I don't really understand what you are looking for?

Pkra added a comment.Nov 7 2018, 11:50 AM

anyone who want's client-side MathJax rendering (for whatever reason) can get it by enabling the source rendering mode (for text browsers) and adding the two lines of configuration you are talking about.

You don't even need to change the renderer. My answer at https://stackoverflow.com/a/31924873/1339651 still works, I think.

@Pkra great. Are we allowed to add this to the [[mw:Extension:Math]]

TheDJ added a comment.EditedNov 7 2018, 1:46 PM

@Physikerwelt that uses external Javascript however. WMF doesn't allow any javascript that is not hosted from within wikimedia (for privacy reasons). The MathJax library code needs to be coming directly from the extension.

With the recent JS attacks and the CSP protection in the process of being added to counter those attacks, likely that referred to snippet will actually stop working somewhere within the next year (because it loads a library from an untrusted source).

Pkra added a comment.Nov 7 2018, 1:49 PM

@Pkra great. Are we allowed to add this to the [[mw:Extension:Math]]

If you are asking about licensing, stackoverflow user content is always licensed as cc-by-sa.

@TheDJ How can WMF not allow Javascript? I think users must have the control what software they execute in their browsers. I think that special small group of people that explicitly wants to use the exact method that StackOverflow and others sites use should have the opportunity to do so. The extension generates MathML and SVG, which should be good enough for the majority of STEM users.

Krenair added a subscriber: Krenair.EditedNov 7 2018, 2:31 PM

They allow JavaScript, but there will be CSP protections that will restrict where you can load external resources from. Users aren't careful enough about what external JS they load into wikimedia users' sessions. And even if they were, not everyone's account security is up to it.

Ok. Does that mean that the following line of @Pkra post
mw.loader.load('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js?config=MML_HTMLorMML-full');//load MathJax with a suitable combined config file
might become dysfunctional in the future?

I would expect so, possibly unless T208188 goes through depending on the details.

Re there being a "special small group of people" who want the same quality of rendering that all other mathematics sites on the web have, and being willing to log in and install special Javascript to achieve it: No. What I want is for all Wikipedia readers to achieve that quality of rendering by default.

@DavidEppstein this is out of question. Even father the quality of the SVG / MathML rendering should be superior to client-side MathJax rendering. However, MathJaxs interaction and context menu functionality is something for "special users".

Do you really think the quality of SVG rendering (MathML is a joke, it doesn't even exist on most browsers) is up to the standard of client-side MathJax, let alone surpassing it? I don't.

@DavidEppstein it's not relevant what I think. A priory, I see no logical reason why the quality of server side generated SVG (by the way using MathJax) should be worse than client side generated output using the same software. For MathML one of the most cited mathematical references the DLMF (e.g., https://dlmf.nist.gov/5.2#E1) uses MathML for browsers that support it. However, I have to admit that proof by example is not a strong argument.

@Physikerwelt For example in the mobile view:

wikipedia server-side svg:

mathjax user script with client-side svg:

Also, stuff like <math>\text{ഗ്ദ്ധ്രീ}</math> doesn't get rendered properly with server side svg rendering. Maybe those things can be fixed, but then we are still far away from having the full functionality like line-breaking, references etc. that you get with normal mathjax and we still have accessibility issues like T194768. I can accept if you don't get those nicely wrapped equations, but something which should always work is printing the source-code. With the client-side JS approach you can be sure that you always get at least the source code, because if for some reason the JS is not loaded you see it and otherwise you can right-click and select source-code rendering in the context menu.

The misaligned mis-sized rendering of Debenben's server-side example, and the inability to properly render \text, are a big part of why so many Wikipedia mathematics editors avoid math mode altogether and use template hacks instead. (Another issue not mentioned by Debenben is that when math is included in a wikilink or external link, as it might be for instance in the title of a mathematics reference, it does not end up colored to make the link visible.) We need to aim for something that is actually uniformly at least as good as current client-side rendering results (regardless of whether it is actually client-side or server-side) in order to make math mode more widely adopted. Sticking our heads in the sand and declaring that the current svg render is already perfect (when it manifestly is not) does not advance that cause.

Izno added a subscriber: Izno.Nov 8 2018, 12:34 AM

Sticking our heads in the sand and declaring that the current svg render is already perfect (when it manifestly is not) does not advance that cause.

Neither does saying "it must be done this way", without apparent regard to the technical requirements that the WMF is levying on you and which Physikerwelt has already made clear to you.

Did you miss the part where I said "regardless of whether it is actually client-side or server-side"? Because saying "it must be done this way" is so far from what I actually said that it appears you are responding to a straw figure of your imagination rather than to my comment.

At a minimum, a good solution must react to the choice of fonts and font sizes that are presented to the reader by the browser for text, and make compatible choices of font, font size, and alignment for the formulas. The complication is that the server cannot know the actual font that will be presented, because of the way CSS allows the browser a sequence of fallback choices for font. But this is not an insurmountable problem for server-side solutions. One could imagine doing this client-side (as most mathjax installations do), or doing a server-side construction of html code that puts the right characters in the right fonts in the right places, or doing a server-side construction of an svg file that puts the right characters in the right fonts in the right places as svg text objects (taking advantage of the fact that text in an svg file is controlled by the same css as text in html). For that matter, if MathML worked, it would also do this, but I think we must give up hope of it ever working on a widespread basis. The solution that doesn't work as well and as far as I can tell will never work as well is the one we have now in which the server makes all the choices and renders the formula as a png or svg image.

Bawolff added a subscriber: Bawolff.Nov 8 2018, 6:52 AM

Ok. Does that mean that the following line of @Pkra post
mw.loader.load('https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.4/MathJax.js?config=MML_HTMLorMML-full');//load MathJax with a suitable combined config file
might become dysfunctional in the future?

Yes. Most definitely.

Basically, if its something we want to support, than Math extension should but the JS on to our servers (assuming licensing is ok) and load via resource loader.

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage.

Pkra added a comment.EditedNov 8 2018, 8:09 AM

@Debenben what browsers those screenshots done with? Such a bad rendering shouldn't happen with the current solution.

Overall, it might help to pick a suitable page and build samples that apply different rendering solutions.

Again, I'd suggest to have a face-to-face meeting. This discussion continues to go in circles and I suspect some direct interaction might help. Maybe using a W3C MathOnWeb CG meeting could be helpful for providing some space for discussions with a wider input.

Pkra added a comment.Nov 8 2018, 8:09 AM

@Bawolff

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage

That was all that was suggested earlier. Thanks for clarifying!

TheDJ added a comment.Nov 8 2018, 8:59 AM

@Bawolff

If its not something "officially" supported, then the relavent JS can go somewhere in the mediawiki namespace, or a JS user subpage

That was all that was suggested earlier. Thanks for clarifying!

Note that you cannot host fonts in mediawiki or user subpages however. So you would have to install those locally, or again fallback on integrating with resource loader.

Pkra added a comment.Nov 8 2018, 11:52 AM

Note that you cannot host fonts in mediawiki or user subpages however. So you would have to install those locally, or again fallback on integrating with resource loader.

Right. Of course people could start inlining the fonts as base64 in a stylesheet. As long as inline style attributes are not blocked, it should be possible to make MathJax work (we're still a bit away from equation rendering without inline styles).

Maybe wecould convince commons to allow font uploads

I think the MathJax fonts are licensed under the SIL Open Font License — at least that's what https://github.com/mathjax/MathJax/issues/1856 from a year ago implies they should be licensed under. However, that license has a no-commercial-redistribution clause (you can't sell the fonts) which may conflict with the sorts of licenses commons allows.

@Debenben what browsers those screenshots done with? Such a bad rendering shouldn't happen with the current solution.

The screen-shot was taken on my home computer (debian stretch, firefox deactivated mathml, open article, scroll down to the very bottom and click on mobile view). At work I did a quick test with windows 10 / firefox, internet explorer, android tablet / samsung & other mobile browsers. Without special plug-ins or user-scripts I could not find a single device /configuration where the current rendering of the mobile view:

https://de.m.wikipedia.org/wiki/Maxwell-Gleichungen#Makroskopische_Maxwell-Gleichungen

is significantly better. I think more than 50% of readers get the mobile view, so this example shows what it currently looks like for the majority of readers.

Neither does saying "it must be done this way", without apparent regard to the technical requirements that the WMF is levying on you and which Physikerwelt has already made clear to you.

What do you mean by "technical requirements that the WMF is levying on you?". I am not aware of any technical requirements other than the JS requirements @TheDJ mentioned above where I cannot see a technical blocking point. Nobody from WMF has come forward with any requirements so far, I think @Bawolff is the first one actively participating. Restrictions for user-scripts are irrelevant because the solution must be something that everybody gets by default. If everybody viewing a page would be downloading the fonts from commons you could immediately break the math rendering on all projects with accidental changes to the font-files and I am not sure how the commons servers would handle the additional load.

Pkra added a comment.Nov 9 2018, 1:36 PM

The screen-shot was taken on my home computer (debian stretch, firefox deactivated mathml, open article, scroll down to the very bottom and click on mobile view). At work I did a quick test with windows 10 / firefox, internet explorer, android tablet / samsung & other mobile browsers. Without special plug-ins or user-scripts I could not find a single device /configuration where the current rendering of the mobile view:
https://de.m.wikipedia.org/wiki/Maxwell-Gleichungen#Makroskopische_Maxwell-Gleichungen

The mobile view simply has a bug here. I've filed https://phabricator.wikimedia.org/T209148

Pkra added a comment.Nov 9 2018, 1:57 PM

Here's a random idea for interested people to try.

The following snippet

document.querySelectorAll('.mwe-math-fallback-image-inline').forEach(
    (img) => {
fetch(img.src)
    .then(r => r.text())
    .then(text => {
        img.outerHTML = text;
    })
    .catch(console.error.bind(console));
    }
)

replaces the equation img tags with the relevant svg source (note: because I'm lazy I only wrote this for modern browsers).

If anyone gives this a spin, I'd be interested in hearing how it affects rendering. For example, equations should inherit color and text elements in the svg should render better (slightly; it's still using monospace).

He7d3r added a subscriber: He7d3r.Nov 12 2018, 3:59 PM

I was looking today for how I am supposed to format commutative diagrams with the current Wikipedia mathematics support. https://en.wikipedia.org/wiki/Help:Displaying_a_formula#Commutative_diagrams says we should just give up, make our own image files offline, and then upload them to commons. But obviously that's going to be even worse for matching fonts and font sizes than the current Wikipedia mathematics support.
This is something MathJax can support: https://math.meta.stackexchange.com/questions/2324/how-to-draw-a-commutative-diagram
I think we should include our non-support of this functionality (assuming my inability to find documentation means it really is non-supported) as a serious deficiency of our current level of mathematics formatting, and add it to the list of things we need to do better.

Pkra added a subscriber: Zorkow.Nov 26 2018, 9:15 PM

Hi everyone,

Volker Sorge (@Zorkow) and I (@Pkra) have drafted a grant proposal for 2019 at [1] titled "Accessibility of equation rendering: a comparative evaluation".

The gist of it from our summary:

We propose a study to analyze the accessibility of equation rendering techniques on the web. This evaluation will focus in particular on the techniques currently used on Wikimedia sites as well as alternatives that are feasible to adopt for Wikimedia.

It would be very helpful to get your feeback on this proposal!

If you could help spread this further in the community (or have recommendations for us on doing that), that would be very useful as well. And of course if you think you can endorse this proposal, that would be extremely helpful.

Thanks,
Peter

[1] https://meta.wikimedia.org/wiki/Grants:Project/Accessibility_of_equation_rendering:_a_comparative_evaluation

Pkra added a comment.Nov 30 2018, 9:19 AM

@DavidEppstein wrote

I think we should include our non-support of this functionality (assuming my inability to find documentation means it really is non-supported) as a serious deficiency of our current level of mathematics formatting, and add it to the list of things we need to do better.

There's https://phabricator.wikimedia.org/T30258 where @Physikerwelt suggested to add AMScd. (Though perhaps the specific nature of AMScd notation might prove an obstacle for texvc.)

Physikerwelt set Due Date to Dec 30 2018, 11:00 PM.
Physikerwelt edited projects, added User-Physikerwelt; removed Patch-For-Review.
Restricted Application changed the subtype of this task from "Task" to "Deadline". · View Herald TranscriptDec 2 2018, 5:37 PM

To come back to the original intent of this ticket. I think we now have a committee. Maybe we can make it a little bit more formal, to have better chances to actually change things. I drafted a community user page here https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math. Otherwise I would say the "inofficial" commitee is:

@Debenben
@Physikerwelt
@Andreg-p
@SalixAlba
@Pkra
@mhchem
@DavidEppstein
@KaiMartin
@Framawiki
@MGChecker
@WhitePhosphorus
@mobrovac
@Mathmensch

I don't think I have knowledge and time to really contributed, I just subscribed this task because I'm interested in development and improvement of the Math extension.

Pkra added a comment.Dec 4 2018, 1:03 PM

@DavidEppstein regarding commutative diagrams some stray thoughts.

I came across T56213 the other day which is tracking issues related to "Infographics, graph, chart, diagram, plotting, animation support". Since many of the tracked issues appear stuck, it might be an opportunity for communities to work together.

That may, however, require giving up the idea of having specific solutions (i.e., xy, pstricks or tikz).

For example, SVG is the natural medium for diagrammatic content on the web. T40271 suggests adding a general purpose SVG editor that provides raw SVG editing as well. It's not too hard to write SVG (especially in comparison to tikz etc) so perhaps editors might be ok with manually authoring SVGs (if they can pull in <math> content)?

There are also linear syntax options with tooling that might be easier to vet by WMF later on (e.g., DOT, JessieCode) and those might be useful more generally both in terms of the wider community as well as in technical terms (e.g., accessibility).

Pkra added a comment.Dec 6 2018, 9:49 PM

@ all is there community support for T32215?

To be accepted as a user group the following is neccary:

Thank you for your recent application to obtain Wikimedia User Group status and your plans to advance Wikimedia Community User Group Math. Below is the list of criteria you must meet before we can move onto full AffCom review. Any criteria not yet met are explained below:

You have met 4 out of 8 criteria:

At least 3 active Wikimedia editors: CRITERION NOT YET MET
Focus advances Wikimedia: CRITERION MET
Mission aligned with Wikimedia Foundation: CRITERION MET
Compliance with naming guidelines and trademark policy: CRITERION MET
Information about group published on a Wikimedia wiki: CRITERION MET
Plans for activities or efforts to advance Wikimedia projects: CRITERION NOT YET MET
Allows new members: CRITERION NOT YET MET
Two designated contacts for Wikimedia Foundation: CRITERION NOT YET MET

At least 3 active wikimedians
We have noticed that you currently do not meet the minimum requirement of having at least 3 active wikimedians as part of your user group. An active wikimedians for the purposes of a user group application is a user who has at least 300 edits and has had a user account for at least 6 months. Are there other users who are part of your user group who you could add to the user group page and meet these requirements?

Minimum Numbers (Recommendation only):
I do not see any editors listed as interested in the group currently. I imagine you are working to recruit more editors to become members and know that can change quickly. The recommended number is at least 10 members to apply to be a user group. While this is not a requirement, I wanted to check, do you have plans for recruit additional members?

Logo (Notification only):
I do not yet see a logo on your page, while it is not required that you have a logo to be recognized as a user group, please see the logo best practices page to learn which logos you may use for your user group (see: https://meta.wikimedia.org/wiki/Wikimedia_movement_affiliates/Logos_best_practices#wugs)

Planned Activities:
We currently do not see any planned activities with specific dates, such as meetings, events, edit-a-thons, etc. on your user group page. Do you have any planned events in the next few months? If so, please add them to the user group page.

Allow New Members:
It is unclear from your application and user group page how you will allow new members to join. Please be sure to add a note as to how other members may join to your page.

Key Contacts:
Two designated contacts for Wikimedia Foundation - The Affiliations Committee and the Wikimedia Foundation staff from time to time need to get in touch with primary contacts from affiliates for various reasons. Can you please specify which two are the primary contacts for your group?

Once we resolve these points I can advance your application for full committee review.

@SalixAlba We have to change our algorithms to look for math tags. dewikiversity is using {{#tag: math}} via https://de.wikiversity.org/wiki/Vorlage:Math on more than 18 000 pages.

Is someone interested in moving the User Group application forward. I think https://meta.wikimedia.org/wiki/Wikimaps_User_Group is an example for a comparable active usergroup. @Thuvack reminded me a few days ago to complete the draft on https://meta.wikimedia.org/wiki/Wikimedia_Community_User_Group_Math but I won't find time for this in the near future.