I think the <math>0</math> case is quite important, despite being easily worked around. Its incorrect rendering has been causing problems on multiple Wikipedia articles, including at least one featured article. It is a very natural formula that is likely to come up again repeatedly.

At https://en.wikipedia.org/w/index.php?title=Binary_search_algorithm&curid=4266&diff=1039519250&oldid=1039518983, Quale found a workaround: add an extra space inside the math formula.

It is not a cache problem (or if it is, it is a cache problem that has lasted six months!). I am consistently seeing hidden category "Pages that use a deprecated format of the math tags" added for any expression that includes \mathbb{C} (but not for other mathbb letters). This format should not be tagged as problematic. Purging does not clear the error, and I get the error on multiple pages (including in my sandbox on preview).

It seems very unlikely to me that a developer-initiated proposal on meta to potentially override a local consensus of en on article content (how to format dates) will generate light rather than heat.

It is the clear and explicit consensus of en.wiki editors not to allow YYYY-MM dates. Not in what is shown to readers, but also not in the source code, and not in the allowed parameters of citation templates. So whether you think YYYY-MM is better or not is irrelevant: if Citoid did this on en.wiki it would be in error and in violation of the local consensus, and should not be used.

There is a very good reason for NOT generating and not accepting dates like 2009-10 in MOS:DATE and in citation template input: Because we don't know whether that means October 2009 or 2009–2010. If the citoid software has a date in this format for which it does know the correct disambiguation, then it is in that software that the conversion to a valid format must be made, before the disambiguation information is lost.

Note however that this should only be done when the ONLY content of the indented line is a single math block. There will be some instances where a displayed formula drops out of math to add some additional text on the same line, and those will probably need a human editor to figure out what do do with them.

I think this is a good idea. See also recent changes to the en.wikipedia manual of style for mathematics formatting, https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Mathematics#Using_LaTeX_markup, that recommended display=block over :-indentation.

A key sequence in the editor is totally useless for me. Like many Wikipedia editors, I avoid the Visual Editor to the extent possible. So for me at least, more effort into keyboard shortcuts there is wasted effort. If it's not in in wikitext I won't use it.

Physikalerwelt's suggestion of backslashed parens and backslashed brackets would be greatly preferable to new longer and nonstandard tags, both because they're much easier to type and because (unlike the math tags) they're familiar and standard TeX syntax that works in other contexts (LaTeX documents and many other web sites). I think this would be much likelier to catch on and actually change the culture of using colons for displayed math.

Presumably part of the issue is that LaTeX renders on the server and slow-rendering LaTeX could DOS the server. Timeouts could help, but quickly detecting problematic LaTeX would (if implemented well) save more time and make DOS more difficult. There's some discussion of this issue at https://www.usenix.org/system/files/login/articles/73506-checkoway.pdf which, while mostly aimed at actual LaTeX installations (where file I/O could be problematic) also address online viewing situations such as this one.

Is sanity not as simple as checking that all characters are printable-ascii and all macros are from a given whitelist? (Assuming the whitelist doesn't include dangerous stuff like \csname) We don't care that it really be valid LaTeX — invalid coding that merely causes MathJax or whatever to issue an error message is still non-problematic.

Is it not possible to use the existing sanity checker, but then if it accepts the input, discard its output and re-use the input as-is?

I agree with Debenden here. Sanity-checking the TeX code before passing it off to MathJax or whatever other formatter is being used: probably a good idea. Trying to mung that code without having full knowledge of the syntax accepted by LaTeX and/or MathJax: a recipe for bad formatting and worse workarounds by users trying to achieve the formatting they expect. Just don't do it.

Sure — go to https://en.wikipedia.org/wiki/A_Guide_to_the_Classification_Theorem_for_Compact_Surfaces, edit the references section using the source editor, and preview. I don't think the visual editor can handle the reference style of that article easily so I haven't tried that. If it makes a difference I am using the monobook skin.

I agree, semantically these are part of inline text and it's the semantics, not the appearance, that should drive our choice of div vs span. So there should be no div and no p, if it's possible (and I'm sure it is) to get the correct appearance with a span instead.

This issue still happens in the current Android app. E.g. view the Wikipedia article "Pythagorean triple": the mathematical formulas in the lead use serif fonts in the browser (correctly) but are shown sans-serif in the Android app (incorrectly).

I was looking today for how I am supposed to format commutative diagrams with the current Wikipedia mathematics support. https://en.wikipedia.org/wiki/Help:Displaying_a_formula#Commutative_diagrams says we should just give up, make our own image files offline, and then upload them to commons. But obviously that's going to be even worse for matching fonts and font sizes than the current Wikipedia mathematics support.
This is something MathJax can support: https://math.meta.stackexchange.com/questions/2324/how-to-draw-a-commutative-diagram
I think we should include our non-support of this functionality (assuming my inability to find documentation means it really is non-supported) as a serious deficiency of our current level of mathematics formatting, and add it to the list of things we need to do better.

I think the MathJax fonts are licensed under the SIL Open Font License — at least that's what https://github.com/mathjax/MathJax/issues/1856 from a year ago implies they should be licensed under. However, that license has a no-commercial-redistribution clause (you can't sell the fonts) which may conflict with the sorts of licenses commons allows.

At a minimum, a good solution must react to the choice of fonts and font sizes that are presented to the reader by the browser for text, and make compatible choices of font, font size, and alignment for the formulas. The complication is that the server cannot know the actual font that will be presented, because of the way CSS allows the browser a sequence of fallback choices for font. But this is not an insurmountable problem for server-side solutions. One could imagine doing this client-side (as most mathjax installations do), or doing a server-side construction of html code that puts the right characters in the right fonts in the right places, or doing a server-side construction of an svg file that puts the right characters in the right fonts in the right places as svg text objects (taking advantage of the fact that text in an svg file is controlled by the same css as text in html). For that matter, if MathML worked, it would also do this, but I think we must give up hope of it ever working on a widespread basis. The solution that doesn't work as well and as far as I can tell will never work as well is the one we have now in which the server makes all the choices and renders the formula as a png or svg image.

Did you miss the part where I said "regardless of whether it is actually client-side or server-side"? Because saying "it must be done this way" is so far from what I actually said that it appears you are responding to a straw figure of your imagination rather than to my comment.

The misaligned mis-sized rendering of Debenben's server-side example, and the inability to properly render \text, are a big part of why so many Wikipedia mathematics editors avoid math mode altogether and use template hacks instead. (Another issue not mentioned by Debenben is that when math is included in a wikilink or external link, as it might be for instance in the title of a mathematics reference, it does not end up colored to make the link visible.) We need to aim for something that is actually uniformly at least as good as current client-side rendering results (regardless of whether it is actually client-side or server-side) in order to make math mode more widely adopted. Sticking our heads in the sand and declaring that the current svg render is already perfect (when it manifestly is not) does not advance that cause.

Do you really think the quality of SVG rendering (MathML is a joke, it doesn't even exist on most browsers) is up to the standard of client-side MathJax, let alone surpassing it? I don't.

Re there being a "special small group of people" who want the same quality of rendering that all other mathematics sites on the web have, and being willing to log in and install special Javascript to achieve it: No. What I want is for all Wikipedia readers to achieve that quality of rendering by default.

There's a relevant discussion going on currently at https://en.wikipedia.org/wiki/Wikipedia_talk:Manual_of_Style/Mathematics starting from a proposal to shift from recommending that displayed mathematics formulas be formatted as :<math>...</math> to instead recommending <math display=block>...</math>. This proposal appears to be failing to reach consensus. The issue is that (although the display=block version generates semantically cleaner html) too many other things work less well with it, including its ability to allow deeper-than-default indentation and its ability to allow non-mathematical material such as reference footnotes on the same line.

In T195861#4308688, @Pkra wrote:

I'd be very interested in hearing more about the issues people have with the SVG output; in my experience it can reach an quality equivalent to the CommonHTML (if done right), even if both formats have trade offs. For example, fixing the SVG stroke size would be a good idea to fix printing.

Re the newly added goal "improve rendering. Benchmark: As good as or better than w:en:Template:Math for all devices": I thought this was about input formats, not so much rendering? But the math template is a pretty low bar: there's too much that it does badly or not at all. The one thing it does really well is match the size and baseline of the non-math text, so if you mean "benchmark: match the text fonts as well as template:math" then that would be clearer. My goal, though, would be as good as or better than client-side MathJax. (One way to achieve that would of course be client-side MathJax but past history on this causes me to doubt WMF will accept that as a viable solution, and anyway if this can be achieved in other ways then why not?)

Getting rid of the definitions you have marked as problematic seems like a good idea, for the reasons described there. But now that we have data on what expressions exist on the wiki, can we find out how often these problematic definitions are used? It looks like it would be easy enough to replace them all with non-problematic equivalents.

I'm also interested.

In fact, the example I already gave had an issue number as well as a "Winter/Spring" date:

Re: "A local template will be able to do a better job of making these dates the right style for the wiki, better than a random nodejs library will do anyway. ": This is not actually true, because (at least on English Wikipedia) multiple date styles are allowed, and "the right style" is the style that is consistent with what has already been used for the other references in the article. The local templates do not generally have this information, any more than citoid does (because it is outside the template parameters and often not recorded with the special "use dmy dates" templates).. So the way to make the dates the right style is to ask the user which style to use before inserting them, rather than passing the job off to some other software that doesn't have any better idea than citoid what to do. Jonesey's suggestion above of a radio button would work.

Regardless of whether 2017-02 is valid as an ISO date, it is not valid for use on Wikipedia. https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Dates_and_numbers says explicitly "Do not use these formats." So the only acceptable solution for month-year date formats appears to be to bite the bullet and internationalize the month names.

I note that citoid is generating fake dates for journal publications that have non-fake dates whose format is beyond what can be represented as YYYY-MM-DD. For instance doi:10.13110/discourse.37.1-2.0003 has an actual date of "Winter/Spring 2015", but when I do 'curl -LH "Accept: application/x-bibtex" http://dx.doi.org/discourse.37.1-2.0003' I only get back "year = 2015" and of course citoid turns that into the false 2015-01-01. You can't do anything here about sources that don't give you the full data, of course, but that could help explain where some of these dateless publication dates are coming from.

Note that the newly merged T165116 is about ISBN data, not DOI data. Citoid is making up fake months and dates for ISBNs which only have years available. As with the DOI, it should not be generating false data and passing it off as accurate.

This is now also happening with <math> markup inside wikilinks, on en.wiki, in some math rendering modes. Specifically, there is no problem with the default bitmap rendering mode for math, but MathML/SVG-fallback mode causes the math to turn into UNIQ...QINU strings when they are inside wikilinks. I'm not sure whether this was broken long-term or whether recent changes to math have exposed math processing to this bug.

Your answer reflects a fundamental misunderstanding of how mathematics is written. Equations are part of the text, not separate sidebar things, regardless of whether they are displayed inline or as separate display equations. For instance, your sample sentence should probably end with a period, in the displayed equation. That period can be inside the math tags, but a reference can't.

It is I think inaccurate to state that the semantics of ":", as used by Wikipedia editors, is "just plain wrong". The intended meaning of this markup is almost always "indent this block of text", a reasonable thing to do and a reasonably clean way to mark it up. The part that is semantically wrong is the conflation of this markup with ;: description-list semantics, and the use of description lists by the Wikimedia engine to indent blocks of text that semantically should be indented some other way. Regardless of that quibble, this change seems like an improvement: display equation math should be rendered in different ways than inline math and it's important to have a way to code that rather than letting the engine guess which style to use with no way to control it when it guesses wrong.

In T111712#1650174, @Jdforrester-WMF wrote:

In T111712#1650158, @DavidEppstein wrote:

I suppose it's too much to ask that we get \[ ... \] as a wiki-shortcut for block math?

Yes.

I agree with the sentiment of having only the inline or block options, with some wiki-local rendering settings to determine the block formatting. Experience has shown that if you give Wikipedia editors two incompatible formatting options they will use both of them and create inconsistencies, and that seems unnecessary here. I have no strong feeling about whether centered or indented is better for block math in the abstract, but keeping the current indented formatting on Wikipedia for now seems likely to cause the least disruption.

SVG still produces \text in inappropriate fonts. It still produces images that have fonts the wrong size for the surrounding text. It still produces images with misaligned baselines. It still produce images with text that does not look as sharp as the actual article text. It still does not let you copy and paste from the middle of a formula. It still does not get colored appropriately when it is part of a link. In all of those respects it is exactly as bad as the bitmap images that we (as editors and readers rather than browser-to-server-communications-purists) have hated for so long. The only improvement of SVG over bitmaps is that if you temporarily zoom your screen differently from your preference setting it scales better. And please please stop talking as if you're moving towards MathML as a default. You're moving towards SVG as a default, with MathML as an "enhancement" for some, with its own different set of display shortcomings.

With the impending removal of MathJax support (T99369) this may have become moot.

@fredw: we don't have to know about Wikipedia's implementation details to know that MathSciNet, arXiv, MathOverflow etc have all been producing high-quality renderings for years using MathJax and that (for those of us not using MathML browsers) Wikipedia and the Wikipedia developers still seems stuck in the "let's just give them an image of the formula" mode that has worked so badly for so long. The format may have changed from gif to png to svg but the idea is the same.

Couple more missing things I noticed were \binom (and \choose but we should be using \binom instead) and \ell

Re KaTeX support: If added, this would need a fallback for the equations that KaTeX does not handle (it only supports a subset of our math markup). Since both KaTeX and the fallback would sometimes both be visible in the same articles, having the same or very similar textual appearance as KaTeX would be a priority in the fallback. But the only option for achieving the same appearance seems to be MathJax, which the developers seem intent on not supporting or including.

How could anything be easier to integrate than MathJax? One line in your html suffices for the simplest uses:

Re: Additionally, it sounds to me like you would prefer to remove TeX support altogether. Please reassure me that I'm misinterpreting you. That would be the death of mathematics on Wikipedia.

@Physikerwelt: What you are actually proposing is "MathML with fallback to SVG with fallback to PNG" and many more users than just the IE6 holdouts will see the SVG-level fallback. For instance, as a Chrome user, that's what I see. I for one find the SVG fallback ugly enough versus MathJax (as detailed above) that I (as a long-term and high-volume Wikipedia mathematics editor) will likely continue using the workarounds. But it seems nothing will persuade you away from the bad track that you seem determined to continue pushing Wikipedia towards, nor your failure to have any concern for the interests of actual mathematics editors, nor from your efforts to keep MathJax away from Wikipedia.

You notice the "with PNG fall-back images" part of the T78046 outline that you linked to? That is what you're actually proposing as the default. I.e., the same bad solution that we've had for ten years and that has been an embarrassment for at least the last six since MathJax has existed and been shown to work everywhere else. If your solution is "X with fallback to Y" then the actual default is Y, and X is just an enhancement for a subset of the users, not any different in principle than what the MathML code is now.

@Dginev: I don't care if it works in Firefox. It needs to just work, period. MathML is a distraction from that. It is diverting your attention from the actual problem, which is the not-logged-in view. MathML will never be the not-logged-in view and any developer work spent improving MathML is developer work wasted on not addressing the actual issue. We don't need to solve the world's problems to realize that the horrible state of mathematics formula formatting on Wikipedia is a major embarrassment to the project and that this particular change and other recent developer activity have zero chance of improving that situation.

@He7d3r the resolution of T38496 as invalid saddens me. Are we going to be locked into ugly bitmap images for math, editors who refuse to use <math> except where absolutely necessary, and badly formatted mathematics in our Wikipedia articles, forever? MathML is NOT a solution. MathJax has been a working solution at every other major mathematics site for six years now.

Possibly it's worth adding that MathML+SVG has a noticeably worse appearance and usability than MathJax in my browser (using Chrome, so falling back to the SVG view): the characters look visibly fuzzier, the whole formula becomes an atomic element in the browser rather than something you can copy and paste pieces out of, and links that include formulas in their link text (for instance in the title of a journal article reference with a link to the original article) show the formula as black instead of matching the color of the rest of the link. In the big formula example in https://en.wikipedia.org/wiki/Squared_triangular_number#Proofs MathJax gives bigger brackets bigger line weight, while SVG paradoxically gives the bigger brackets smaller weight, and SVG also has significantly less even spacing around the plus signs in that formula. In the inline text in the paragraph immediately below the big formula, the SVG baselines are far from correct.

Re: "Can you be a little bit more specific why MahtML is not good enough?" As I already said, the main reason is because it does not work in Chrome. Instead you have to use a different method, SVG, for that browser. If SVG is good enough to use as a default not-logged-in view, it is good enough, and MathML is not necessary to fix the current problems. If SVG is not good enough then having MathML patch it to make it better on some but not all browsers is not sufficient to fix the problem; we will still be trying to use other markup to work around the rendering deficiencies. So in either case the MathML part is a waste of time and the SVG part is what you should be working on.

Also if this is a fait accompli as the technews piece implies, what is the point of asking the Math project for feedback so late and then failing to respond to the resulting discussion? It just comes across as a deliberate snub and accentuates the past history of non-responsiveness of the Wikimedia developers to actual concerns of Wikipedia editors.

I'm going to repeat here a comment I already made at Wikipedia talk:WikiProject Mathematics to make sure that other developers who might be interested in this issue see it.