Page MenuHomePhabricator

Update mathoid to node 16
Open, Needs TriagePublic

Description

From https://gerrit.wikimedia.org/r/c/mediawiki/services/mathoid/+/808998

Apparently eslint dropped support for node 10 in v8.0.0 (https://eslint.org/docs/latest/user-guide/migrating-to-8.0.0#drop-old-node). And we use node 10 in our production mathoid images, right? So, do we need to continue to use an older version of eslint?

Apart from that, the possibility of running node 10 in production is about to be switched off soon.

To switch to a new node version, we need to

We should also update the libraries in the package.json file. Attention is required for

Event Timeline

Restricted Application added a subscriber: Aklapper. · View Herald Transcript
Physikerwelt renamed this task from Consider using a newver node version for mathoid in production to Consider using a newer node version for mathoid in production.Jul 7 2022, 7:44 AM

@Jdforrester-WMF T247697 is the main problem here. Someone suggested replacing librsvg with https://github.com/yisibl/resvg-js however, this would require quite a bit of testing, and it would make Math even more special, while my goal is to make math something that everyone can maintain without special knowledge. With MathML intending to land in Chrome 109 https://chromestatus.com/feature/5240822173794304 the number of people using images will get smaller eventually. And those not able to use PNG images would have an even better alternative. So if there is no native WMF way to convert the Math SVGs to PNGs I would rather want to fade out PNGs instead of installing yet another special thing.

@Jdforrester-WMF T247697 is the main problem here. Someone suggested replacing librsvg with https://github.com/yisibl/resvg-js however, this would require quite a bit of testing, and it would make Math even more special, while my goal is to make math something that everyone can maintain without special knowledge. With MathML intending to land in Chrome 109 https://chromestatus.com/feature/5240822173794304 the number of people using images will get smaller eventually. And those not able to use PNG images would have an even better alternative. So if there is no native WMF way to convert the Math SVGs to PNGs I would rather want to fade out PNGs instead of installing yet another special thing.

Dropping production code in Node 10 was meant to have fully happened over a year ago.

I don't think waiting around for use of PNGs to drop is the right approach, but also, the support for MathML in Chrome 109 shouldn't matter, as those users can already use the SVGs, surely, and don't need the PNGs? Instead, I'd recommend just killing the SVG->PNG functionality immediately.

Instead, I'd recommend just killing the SVG->PNG functionality immediately.

Can I quote you on this? Then I would do it immediately:-)

If you have access to the database, you can check how many users have the png mode in their user options for the math setting.

Instead, I'd recommend just killing the SVG->PNG functionality immediately.

Can I quote you on this? Then I would do it immediately:-)

Yes.

If you have access to the database, you can check how many users have the png mode in their user options for the math setting.

Not many, from a quick look.

SELECT up_value, COUNT(*) AS count FROM user_properties WHERE up_property = 'math' GROUP BY up_value; for enwiki gives:

up_valuecount
''524
08125
213000
32615
443143
510744
64822
'mathml'3333
'on'26
'png'4967
'source'2355

As ever, most of these are nonsense values, but applying current code in my head we get:

up_valuecountscount
'png' (1)49674967
'source' (2)2355+1300015355
'mathml' (3)3333+524+8125+2615+43143+10744+4822+2673332 (+ all other users)

Ok. I sent out a quick heads-up to the math mailing list. Removing PNG support is easy. However, restbase relies on PNG images from mathoid and thus needs to be changed before. Removing the user preference PNG in the math extension is also easy. I can submit a patch for that.

Change 858661 had a related patch set uploaded (by Jforrester; author: Physikerwelt):

[mediawiki/extensions/Math@master] Remove PNG rendering mode

https://gerrit.wikimedia.org/r/858661

Change 858661 merged by jenkins-bot:

[mediawiki/extensions/Math@master] Remove PNG rendering mode

https://gerrit.wikimedia.org/r/858661

Tacsipacsi added a subscriber: Tacsipacsi.

The removal of a former (before b9f84f804) default option warrants an entry in next Tech News IMO.

The removal of a former (before b9f84f804) default option warrants an entry in next Tech News IMO.

It's not been default for nearly five years. Anyone who used to use it by default changed configuration then, which would have been the time to put in the user notice. But sure.

It is like not producing new music on Musicassette anymore. For most people it's almost a fun fact (Did they really produce MCs for so long?), but there will be that one very special guy...

if they store them in swift, is it safe to get rid of them there too?

Xaosflux added a subscriber: Xaosflux.

Hello, just FYI, we are getting reader complaints at https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#SVG_instead_of_PNG that the removal of PNG option makes content less accessible to readers wanting to reuse our published content.

Hello, just FYI, we are getting reader complaints at https://en.wikipedia.org/wiki/Wikipedia:Village_pump_(technical)#SVG_instead_of_PNG that the removal of PNG option makes content less accessible to readers wanting to reuse our published content.

I read the linked page and did not understand the argument. If there is a regression in the user experience, I am happy to contribute to a fix. I just don't understand the problem. (Coping -> screenshot seems fine to me, but is there anything else.)

FWIW I agree with @Physikerwelt, we can't do everything.

I think their arguments are that dowloading a rendered PNG file are both more accessible and have a higher quality then trying to take a screen capture. This would likely present more if they are trying to copy a section of text intertwined with formulas, as a resulting paste would then be lacking only the formulas.

I think their arguments are that dowloading a rendered PNG file are both more accessible and have a higher quality then trying to take a screen capture. This would likely present more if they are trying to copy a section of text intertwined with formulas, as a resulting paste would then be lacking only the formulas.

I can understand that. I filed a task for this specific issue T324451.

Can we migrate directly to node 16 (skipping 12 and 14)?

Yes, if you can do all the changes in one go https://docker-registry.wikimedia.org/nodejs16-slim/tags/ is available right now.

Change 865787 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/mathoid@master] Remove PNG support

https://gerrit.wikimedia.org/r/865787

Physikerwelt renamed this task from Consider using a newer node version for mathoid in production to Update mathoid to node 16.Dec 10 2022, 8:21 AM
Physikerwelt updated the task description. (Show Details)

Change 866662 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/mathoid@master] Update svgo

https://gerrit.wikimedia.org/r/866662

Change 865787 merged by jenkins-bot:

[mediawiki/services/mathoid@master] Remove PNG support

https://gerrit.wikimedia.org/r/865787

Change 865787 merged by jenkins-bot:

[mediawiki/services/mathoid@master] Remove PNG support

https://gerrit.wikimedia.org/r/865787

This should now be deployed to beta (e.g. https://en.wikipedia.beta.wmflabs.org). @Physikerwelt

Change 865787 merged by jenkins-bot:

[mediawiki/services/mathoid@master] Remove PNG support

https://gerrit.wikimedia.org/r/865787

This should now be deployed to beta (e.g. https://en.wikipedia.beta.wmflabs.org). @Physikerwelt

It seems to have broken beta as you predicted :). Error message:

Failed to parse (SVG (MathML can be enabled via browser plugin): Invalid response ("Math extension cannot connect to Restbase.") from server "/mathoid/local/v1/":): {\displaystyle f(z) = \sum_{n=0}^{\infty} a_n (z - z_0)^n}

You see the error if you contact RestBase directly,

curl -vd 'q=E=mc^2 \text{221220}' https://en.wikipedia.beta.wmflabs.org/api/rest_v1/media/math/check/tex

{"type":"https://mediawiki.org/wiki/HyperSwitch/errors/internal_http_error","method":"post","detail":"getaddrinfo ENOTFOUND deployment-docker-mathoid02.deployment-prep.eqiad.wmflabs deployment-docker-mathoid02.deployment-prep.eqiad.wmflabs:10044","uri":"/en.wikipedia.beta.wmflabs.org/v1/media/math/check/tex"}%

It seems something is misconfigured?

(Note that you need to use a new query string whenever you make a successful attempt to bypass caching.)

Change 869742 had a related patch set uploaded (by Ladsgroup; author: Amir Sarabadani):

[operations/mediawiki-config@master] Labs: Use the correct address for mathoid

https://gerrit.wikimedia.org/r/869742

Change 869742 merged by jenkins-bot:

[operations/mediawiki-config@master] Labs: Use the correct address for mathoid

https://gerrit.wikimedia.org/r/869742

That needed fixing regardless but I still need to figure out where that's coming from because it's trying to connect to mathoid02 (correctly, just FQN is wrong) while the mw-config had on mathoid01. I should dig in HyperSwitch.

The value in restbase04 in /etc/restbase/config-vars.yaml is correct. I restarted it and that I think fixed it.

Thank you. The deployment is fixed. However, future changes in restbase seem to be necessary. See the current output https://en.wikipedia.beta.wmflabs.org/api/rest_v1/media/math/render/svg/de352569c2c921965ffe883b3eb11f48d4af300f

{
  "type": "https://mediawiki.org/wiki/HyperSwitch/errors/server_error#empty_response",
  "method": "get",
  "detail": "Math: missing or malformed response for format png",
  "uri": "/en.wikipedia.beta.wmflabs.org/v1/media/math/render/svg/de352569c2c921965ffe883b3eb11f48d4af300f"
}

I see a similar error when I ask for a png, e.g. https://en.wikipedia.beta.wmflabs.org/api/rest_v1/media/math/render/png/61b3082f39193dd9aa038a8f7d8cafbdb5103db4

I am also seeing errors in logstash, but they don't always get shown to the user.

I occasionally see errors when creating/editing maths formulae in VisualEditor. May be worth testing further.

Change 869763 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/restbase@master] Remove pngs from math endpoint

https://gerrit.wikimedia.org/r/869763

I still think we need to adjust restbase along the lines of https://gerrit.wikimedia.org/r/c/mediawiki/services/restbase/+/869763/. However, it was 2016 when I made the last change to restbase (https://github.com/wikimedia/restbase/commit/4fccb85ffaf154be3ef12aa684c2a8172a2d9d62). At that time, restbase was stored on GitHub.

Change 866666 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/mathoid@master] Update dependencies and require node16

https://gerrit.wikimedia.org/r/866666

Change 873490 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/extensions/MathSearch@master] Remove png support

https://gerrit.wikimedia.org/r/873490

Change 873490 merged by jenkins-bot:

[mediawiki/extensions/MathSearch@master] Remove png support

https://gerrit.wikimedia.org/r/873490

@Physikerwelt I have managed to get both Mathoid and Restbase running locally.

I am testing out https://gerrit.wikimedia.org/r/869763.

After checking it out, I no longer get the error (printed to stdout by the restbase service):

{"name":"restbase","hostname":"x200","pid":2757,"level":50,"message":"500: server_error#empty_response","res":{"name":"HTTPError","message":"500: server_error#empty_response","status":500,"body":{"type":"server_error#empty_response","description":"Math: missing or malformed response for format png"}, ...

which is good.

However, I am getting a new error when loading a path with <math> tags:

{"name":"restbase","hostname":"x200","pid":2822,"level":40,"err":{"message":"404: not_found#route","name":"restbase","stack":"HTTPError: 404: not_found#route\n    at HyperSwitch._request (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:270:19)\n    at _createFilteredHandler (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:172:26)\n    at tryCatcher (/home/drw/wikimedia/srv/restbase/node_modules/bluebird/js/release/util.js:16:23)\n    at /home/drw/wikimedia/srv/restbase/node_modules/bluebird/js/release/method.js:15:34\n    at handlerWrapper (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:424:37)\n    at next (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:410:42)\n    at Object.module.exports [as filter] (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/filters/http.js:24:16)\n    at handlerWrapper (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:422:27)\n    at /home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:428:28\n    at HyperSwitch._filteredRequest (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:173:19)\n    at HyperSwitch.(anonymous function) [as post] (/home/drw/wikimedia/srv/restbase/node_modules/hyperswitch/lib/hyperswitch.js:462:21)\n    at MathoidService._invalidateCache (/home/drw/wikimedia/srv/restbase/sys/mathoid.js:187:22)\n    at MathoidService._storeRenders (/home/drw/wikimedia/srv/restbase/sys/mathoid.js:149:24)\n    at hyper.post.then (/home/drw/wikimedia/srv/restbase/sys/mathoid.js:166:18)\n    at tryCatcher (/home/drw/wikimedia/srv/restbase/node_modules/bluebird/js/release/util.js:16:23)\n    at Promise._settlePromiseFromHandler (/home/drw/wikimedia/srv/restbase/node_modules/bluebird/js/release/promise.js:547:31)","status":404,"body":{"type":"not_found#route","title":"Not found.","internalURI":"/wikimedia.org/sys/events/","method":"post","depth":2},"root_req":{"method":"get","uri":"/localhost/v1/media/math/render/svg/6e865e5b244a7a5d4fdf7a95fe4cea6d25e581ac","headers":{"cache-control":"no-cache","user-agent":"Mozilla/5.0 (X11; Linux ppc64le; rv:91.0) Gecko/20100101 Firefox/91.0","x-client-ip":"::ffff:192.168.10.117","x-forwarded-for":"::ffff:192.168.10.117","x-request-id":"2971a6e0-9270-11ed-9678-6fffb2fa919e","x-request-class":"external"}},"request_id":"2971a6e0-9270-11ed-9678-6fffb2fa919e","api_path":"/{domain}/v1/media/math/render/{format}/{hash}","levelPath":"warn/bg-updates"},"msg":"404: not_found#route","time":"2023-01-12T11:56:34.906Z","v":0}

But, in spite of this, on my local mediawiki using my local restbase the formulae display fine and no signs of anything going wrong. I just get the above error printed by restbase.

My setup is:

However, I am getting a new error when loading a path with <math> tags:

This error might be T216433. I tried to modify my config.yaml in the way suggested by T216433#4966821 but I couldn't get it to work.

Hi @dom_walden,

Thank you for taking the time and testing the change. I propose, instead of testing from the math extension, testing from restbase. Otherwise, there are too many intermediate steps.

Are you aware of

https://gerrit.wikimedia.org/r/c/mediawiki/services/mathoid/+/875405

from @daniel ?

I guess if you test that, you should be able to get the SVG and MathML from Restbase but no longer PNG. Does that work?

If so, the next step would be to check if you use your local restbase server from your math extension. The error message, i.e.,

{"type":"not_found#route","title":"Not found.","internalURI":"/wikimedia.org/sys/events/","method":"post","depth":2}

suggests to me that this is unrelated to https://gerrit.wikimedia.org/r/869763. So you should get the same error message wether or not you try with master or https://gerrit.wikimedia.org/r/869763 or restbase. I think the config provided by @daniel solves the localhost vs. wikimedia.org issue.

Hi @dom_walden,

Thank you for taking the time and testing the change. I propose, instead of testing from the math extension, testing from restbase. Otherwise, there are too many intermediate steps.

I have given the restbase API a brief test. I could use the three endpoints (/media/math/check/tex, media/math/formula/$hash, /media/math/render/$format/$hash) to generate an SVG and MML representation of a formula.

/media/math/render/png/$hash just returns: "data.params.format should be equal to one of the allowed values: [svg, mml]"

Are you aware of

https://gerrit.wikimedia.org/r/c/mediawiki/services/mathoid/+/875405

Yes, that was what I was using to get my local setup working.

I guess if you test that, you should be able to get the SVG and MathML from Restbase but no longer PNG. Does that work?

Yes. Locally, maths formulae display fine.

I tried with different values of $wgDefaultUserOptions['math'] and they all worked (png just returned an MML+SVG).

If so, the next step would be to check if you use your local restbase server from your math extension. The error message, i.e.,

{"type":"not_found#route","title":"Not found.","internalURI":"/wikimedia.org/sys/events/","method":"post","depth":2}

suggests to me that this is unrelated to https://gerrit.wikimedia.org/r/869763. So you should get the same error message wether or not you try with master or https://gerrit.wikimedia.org/r/869763 or restbase. I think the config provided by @daniel solves the localhost vs. wikimedia.org issue.

I was using their config. But I think you are right that it is unrelated. I don't think it needs to be investigated further.

Change 866666 had a related patch set uploaded (by Physikerwelt; author: Physikerwelt):

[mediawiki/services/mathoid@master] Update dependencies and require node16

https://gerrit.wikimedia.org/r/866666

I have also tested this in my local mediawiki + mathoid + restbase setup. I have not seen any problems. Maths formulae display fine.

Thank you. What is the next step? Can I do anything to foster progress?

Change 869763 abandoned by Physikerwelt:

[mediawiki/services/restbase@master] Remove pngs from math endpoint

Reason:

review on github

https://gerrit.wikimedia.org/r/869763

Thank you. What is the next step? Can I do anything to foster progress?

Sorry for the late reply.

Hopefully @Jgiannelos can review https://github.com/wikimedia/restbase/pull/1311.

I will look into getting it deployed to beta as well as https://gerrit.wikimedia.org/r/c/mediawiki/services/mathoid/+/866666.

Thank you. The review for https://gerrit.wikimedia.org/r/c/mediawiki/services/mathoid/+/866662/7 was already done; however, CI-merge failed. I think a consultation with OPS is required here.