Remove PNG only rendering mode
Closed, ResolvedPublic

Description

To be more flexible with new features the PNG only rendering mode should be deprecated and finally be removed.

PNG only rendering mode was deprecated on

  • 2014-05-28

I suggest to disable it in production in

  • 2017-02-01

and to remove the code from the math extension in

  • 2017-03-01

Details

Reference
bz72240

Related Objects

There are a very large number of changes, so older changes are hidden. Show Older Changes
Prod added a subscriber: Prod.Jul 20 2015, 10:28 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 20 2015, 10:28 PM
Paladox set Security to None.Jul 26 2015, 9:53 AM
Paladox added a subscriber: Physikerwelt.
Restricted Application added a subscriber: StudiesWorld. · View Herald TranscriptNov 29 2015, 2:06 PM
He7d3r added a subscriber: He7d3r.

Should this be high priority since maths tests are broken.

I\m not sure I understand the purpose of this task. If there's no RB/Mathoid, then we can only have the PNG mode. On the other hand, we might instruct administrators that cannot or do not wish to install the extra services to point their configs to our production environment.

The point is the following:
If we introduce new features, we do not want to make them backwards compatible with the old png rendering mode.
\ce for instance is an example of a command that does not work in png rendering mode.

By "PNG only" you mean texvc correct? What are the other alternatives besides mathoid if png/texvc is removed as an option?

source rendering mode

Legoktm added a comment.EditedDec 23 2016, 12:51 PM

That's not really a graphical rendering mode is it? If I look at the pythagorean theorem article with source mode, I see $ a^{2}+b^{2}=c^{2}, $, which I suppose for some people is understandable, but I don't think most casual math users/readers can follow that when it gets more complicated.

I'm asking because we just spent a decent amount of effort to get texvc back into Debian (https://blog.legoktm.com/2016/12/23/texvc-back-in-debian.html), but if it's going to be gone in 3 months, then we need to re-evaluate.

I think it would help if you could articulate the problems with supporting png mode. Is it lack of a maintainer to support updates needed for texvc? I think people are going to find setting up restbase and mathoid difficult, so it would be nice if there was something simpler that provided graphical rendering (of course, requiring an ocaml binary isn't simple, but I think with OS packages, it's easier than setting up a network service).

@Legoktm thank you for maintaining the texvc package. Do you think it could be simply swapped of with the mathoid/restbase package? @GWicke what do you think?

@Legoktm thank you for maintaining the texvc package. Do you think it could be simply swapped of with the mathoid/restbase package? @GWicke what do you think?

It is definitely possible, but needs a lot more investigation, and will definitely not make it in time for Debian Stretch. But, I still would like to see my question from above answered:

I think it would help if you could articulate the problems with supporting png mode. Is it lack of a maintainer to support updates needed for texvc?

@Physikerwelt and I discussed this at the dev summit, and one of the ideas that came out of it was T155201: Support shelling out to mathoid instead of requiring a full server/restbase setup.

Arthur2e5 added a subscriber: Arthur2e5.

@Legoktm the node only math rendering is available now. Should we continue our discussion on a mediawiki-math debian package?

@Legoktm the node only math rendering is available now. Should we continue our discussion on a mediawiki-math debian package?

Yes :) So I updated the dependency list for mathoid...and it got much larger: https://www.mediawiki.org/wiki/User:Legoktm/Packages/mathoid

If this is CLI only, does it still need dependencies like swagger-ui, service-runner, and so on?

If this is CLI only, does it still need dependencies like swagger-ui, service-runner, and so on?

It's not CLI-only. It's the regular Mathoid service that includes a CLI mode, so yes, these are needed.

@mobrovac in theory we could split mathoid in two packages mathoid and mathoid-core to get rid of the swagger ui dependency. However, I think with mathjax, mathjax-node, texvc, tevcinfo and mathoid we alredy have to maintain quite a few packages.

Exactly, and it wouldn't bring us much benefit other than getting rid of a couple of dependencies...

Exactly, and it wouldn't bring us much benefit other than getting rid of a couple of dependencies...

According to https://www.mediawiki.org/wiki/User:Legoktm/Packages/mathoid removing swagger-ui alone would remove 313! packages from the dependency tree. That's a ton of packaging work for no need (to the point where I wouldn't even waste my time trying). Furthermore removing unnecessary code reduces attack surface, etc. In general I like less code to worry about :-)

@mobrovac in theory we could split mathoid in two packages mathoid and mathoid-core to get rid of the swagger ui dependency. However, I think with mathjax, mathjax-node, texvc, tevcinfo and mathoid we alredy have to maintain quite a few packages.

Most of those have one or no dependencies - that's managable.

@Legoktm can you elaborate on the process of creating a Debian package, so that one can estimate how much effort each choice causes.

Arthur2e5 added a comment.EditedMar 2 2018, 3:02 PM

The "proper" way of creating a debian package involves making a "debian" directory in the source and writing rules in it. There's a control file that gives package name and dependencies, a big Makefile called "rules", and a changelog for versioning to write. Debian likes to take extra care to split packages into smaller ones, but you don't have to do that.

Writing one from scratch might take one or two hours given how boring it is, but you can always reuse what they have in those "debian" tarballs for packages and adapt them to your case…


If you want to do it the crude way, all you need is really just to write the control, do a make install to a destdir, put the control under $DESTDIR/DEBIAN, and run dpkg-deb or something on the DESTDIR.

Right, but packages containing other dependencies inside them will not be accepted by upstream.

Maybe we can make the use of swagger-ui optional in Mathoid's code and then just not install/require it in the Debian package?

You can make your own repositories! Just clump some deb packages in one place, write a Release file, and let dpkg-scanpackages generate a Package index.

The goal is that people can just apt-get install mediawiki-math as they could in the past. I'm not sure if setting up own packages is easier than just installing node and to use npm i mathoid.

I don't think using an alternative repo is an option, really, because we want eventually for the mediawiki package to depend on it (@Legoktm correct me if I'm wrong).

The goal is that people can just apt-get install mediawiki-math as they could in the past. I'm not sure if setting up own packages is easier than just installing node and to use npm i mathoid.

This is a good point too :)

Change 415879 had a related patch set uploaded (by Mobrovac; owner: Mobrovac):
[mediawiki/services/mathoid@master] Make swagger-ui optional

https://gerrit.wikimedia.org/r/415879

@Legoktm can you elaborate on the process of creating a Debian package, so that one can estimate how much effort each choice causes.

The technical steps are pretty simple, I think they're actually mostly automated for nodejs. The biggest amount of work is doing copyright audits, and ensuring compliance with other Debian policies (builds from source, etc.). I estimate it is about 2-3 hours per small package. Then my sponsor has to also review the package, and then upload it for review by the Debian FTP team :)

You can make your own repositories! Just clump some deb packages in one place, write a Release file, and let dpkg-scanpackages generate a Package index.

Wikimedia already has its own repository to provide parsoid packages, but the goal is to allow Debian users to just apt install mathoid like they currently can for texvc.

The goal is that people can just apt-get install mediawiki-math as they could in the past. I'm not sure if setting up own packages is easier than just installing node and to use npm i mathoid.

Unfortunately, npm is a disaster in Debian :( See https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=857986 for the sad details. Additionally, my understanding is that some people are using the Debian package in a firewalled environment and might not be able to use npm. (Similar to how Wikimedia probably isn't going to just npm i <whatever> in production for some upstream software we want to use :))

I don't think using an alternative repo is an option, really, because we want eventually for the mediawiki package to depend on it (@Legoktm correct me if I'm wrong).

Probably not depend, but recommend or suggest, yeah :)

I think the direction of making swagger-ui optional makes a huge difference. Without it, I think we're down to ~50 packages, which is a lot more reasonable.

Change 415879 merged by Mobrovac:
[mediawiki/services/mathoid@master] Make swagger-ui optional

https://gerrit.wikimedia.org/r/415879

Krinkle removed a subscriber: Krinkle.

@Legoktm Good news: @mobrovac merged the change to make swagger-ui optional. Do you think it's now doable to make the debian package, so that we can eventually remove texvc, ocaml, latex and so on from production.

Physikerwelt raised the priority of this task from Normal to High.Apr 21 2018, 1:43 PM

This bug blocks basically any changes related to Math. For instance the unit tests are no longer executed, which makes me somehow nervous. My attempt to re-enable them failed, since texvc / texvccheck is not available in the new test environment log.

What is there left to do here? Disable Math extension's PNG-only mode and remove it form its code-base?

Since Mathoid can also do PNG, I don't see a problem in doing this.

The discussion above suggested that we should offer an alternative Debian package. I could take care of removing the PNG and latexml rendering mode.

I have discussed this issue again with @mobrovac today. We will keep the PNG mode for now, but serve the PNG images from mathoid.

I have discussed this issue again with @mobrovac today. We will keep the PNG mode for now, but serve the PNG images from mathoid..

@Physikerwelt it should be a simple Math ext change, correct?

@mobrovac simple but not trivial. We could introduce another class (e.g., MathPNG) based on the MathML class that overwrites the output method only. This class would always deliver png rather than mathml + svg fallback.

@mobrovac simple but not trivial. We could introduce another class (e.g., MathPNG) based on the MathML class that overwrites the output method only. This class would always deliver png rather than mathml + svg fallback.

Sound like a good first step. We could then eventually phase out the old PNG renderer class to simplify the code.

Change 434048 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/Math@master] Remove old texvc files

https://gerrit.wikimedia.org/r/434048

Change 433750 had a related patch set uploaded (by Physikerwelt; owner: Physikerwelt):
[mediawiki/extensions/Math@master] Serve png mode from mathoid

https://gerrit.wikimedia.org/r/433750

Change 433750 merged by jenkins-bot:
[mediawiki/extensions/Math@master] Serve png mode from mathoid

https://gerrit.wikimedia.org/r/433750

Kulla removed a subscriber: Kulla.Tue, May 22, 10:26 AM
Physikerwelt closed this task as Resolved.

This is done. Let's keep track of the cleanup process in T195847.