Hopefully it doesn't take another 13 years.

I'd just like to add that a year and a half later, using the ImageMagick 7.1.1-43 implementation in Debian Trixie, encoding AVIF is much much faster (but HEIF encoding is still slow). Using the same typical large photograph as last time (extracted PNG from a Canon CR3 raw file):

Thank you - improving PDF output on T181322 also depends on this.

In T385344#10885471, @Jdlrobson-WMF wrote:

I believe this could be handled by a transform.
https://www.mediawiki.org/wiki/Extension:Chart/Transforms

I want to test adding one woff2 file to the static resources of the Wikisource extension, which adds it to the static resources pipeline for deployment, and takes care of not having to add it to ULS, or pollute all of the other sister projects with a font that isn't needed outside this particular specialist usage on Wikisource. Petit Formal Script is open source with a SIL license, so we don't have to source it from Google or any other random CDN. The font designer Pablo Impallari has published all of them on GitHub. Except, of course, for one font. Can you guess which one? Correct! Petit Formal Script. So I've emailed him to see what the story is. In the meantime I am testing with a woff2 conversion of the TTF file supplied in the Google download zip. For now I am also not bothering with other webfont formats since woff2 is now supported everywhere.
Update: source for Petit Formal Script is at github.com/librefonts/petitformalscript

No further action required, this behaviour is in Lilypond not the Score extension, has a well known workaround as explained by @Beeswaxcandle, and is fixed anyway in Lilypond 2.24 which will hopefully be deployed soon via T385404.

No worries, and thanks again 🙂

Well 2.24 is the current stable releases, and 2.25 is the dev stream. I'm not sure if it's a good idea to use 2.25 until there's at least an RC, or a 2.26 stable release? Sorry, I'm not familiar enough with what the 2.25 new features are, or breaking changes (if any).

Thank you, done: T385404

Is anyone actually paying any attention at all to this bug, or what?

I have spare evenings here and there, I'd be happy to muck in if there's some code I can push around on Gerrit?

So, to solve this for Wikisource instances where use of "cursive" font family is occasionally required, we should add the Petit Formal Script woff2 to the default theme payload, or something?

Right. So since it can be solved in the same way, why has it taken seven years to not add a woff2 file in ULS, is it because the list of fonts provided can't be configured per wiki instance (e.g. include it in Wikisource config, but not others)?

If it's not a silly question, could this be solved in the same manner as the font used for the Wikisource Blackletter template?

I also wonder if we're trying to do much. It's already evident we don't have enough engineering resource to move some of these tickets. If the goal is to serve SVG files to be rendered in the client, then we should probably stick to SVG 1 and if there are weird text issues, then it is incumbent on the uploader to fix - use a common font, convert text to shapes, etc. I'm just an punter not an employee, so maybe I'm wrong, but it seems to me that the overall scope is ultimately to provide vector diagrams to illustrate an encyplopædia. Maybe we don't need to get too fancy :) It sure would be nice to have SVG music snippets T49578 for example, some time this decade.

In T49578#9915561, @Skierpage wrote:

The choices seem to be:

• Create a Debian 12 (Bookworm) shellbox to run the Debian Lilypond 2.24 package modified with cairo support.
• Backport the Debian Lilypond 2.24 with cairo support to Buster. (I tried building this in a Debian Buster container and ran into a debhelper-compat issue before even getting to the guile problems @Legoktm encountered)
• "The last resort" discussed above: run the Lilypond 2.24.3 generic Linux download from upstream in current shellbox. It worked for me in my Debian Buster container without requiring any install steps or moving files around, but I wasn't using shellbox.

In T40010#9653241, @Tercer wrote:

In a recent discussion in WikiProject Mathematics yet another rendering bug was encountered. Several users expressed the sentiment that SVG support in MediaWiki will never get fixed, and it is better to give up on them altogether and revert to PNGs. This is specially frustrating because the browsers can render the SVG correctly, but MediaWiki insists on passing it through librsvg and serving the resulting garbage instead.

Hi, nothing's happened for a week now? https://quarry.wmcloud.org/query/84387

I am now down a rabbit hole updating someone's BRISQUE, PIQE and NIQE code. It mostly now seems to work, but I don't have an NVidia card to test the BRISQUE code on.

In T257652#9993075, @Trougnouf wrote:

WebP is 8-bit only, right? (The reason I need a modern codec like AVIF/HEIC/JPEG XL is higher color bit depth.)

That is very cool :) My only nit-pick might be that starting with a JPEG source image might bias or throw off the lossy compression, since it will already have minor block and ring artifacts (especially if you're measuring SSIM or PSNR). Might be better to find an uncompressed TIFF or RAW image. Also, it might be fun to compare HDR images with higher bits (10 or 12 per element, e.g. RAW) which is where the newer WebP and AVIF codecs do a better job; and I'd like to dig into things like PIQE or BRISQUE measurements if I get time (unlikely!) since SSIM is a bit blunt.

Apologies - for what it's worth, using an arbitrarily chosen but reasonable quality of 80% for both conversions:

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert -quality 80 IMG_0652.png IMG_0652.avif
real    0m30.996s
user    1m49.326s
sys     0m3.958s
$ time convert -quality 80 IMG_0652.png IMG_0652.jpeg
real    0m0.858s
user    0m0.721s
sys     0m0.136s
$ ls -l
-rw-rw-r-- 1 --- --- 1.1M Jul 17 16:19 IMG_0652.avif
-rw-rw-r-- 1 --- --- 2.4M Jul 17 16:19 IMG_0652.jpeg
-rw-rw-r-- 1 --- ---  41M Aug 24  2023 IMG_0652.png

I'm not sure that "80%" means the same thing in the different codecs, however.

Hi, strangely, some of my MIDI uploads have not been transcoded (Quarry) e.g. in my list of Commons uploads, the first six are fine, but File:6-Z46B_set_class_on_C.mid and on are still not, after 3 days?

Hi, apologies that I don't know how to help, and not sure if it's the same problem; the manual intervention here worked for files I uploaded to Commons on Monday (e.g. File:3-5B set class on C.mid), but files I uploaded yesterday are still waiting, e.g. File:4-3 set class on C.mid. I made a quarry query for these.

In T49578#8708363, @Legoktm wrote:

Our standard practice is to prefer using Debian packages when possible

@Legoktm hi - currently this is not possible due to the Debian maintainer leaving out libcairo support for some (undocumented as far as I can tell) reason.

As predicted, utter radio silence from the Debian maintainer of the 2.24 Lilypond package, so what do we do now? Make some noise on the Debian bug so the next version of Debian due in a couple of years might have libcairo built-in? Or can we please just get on with it, and use the generic binary from upstream Lilypond, which works fine on my Bookworm test instance? I'm not in a position to "deploy" any of this anywhere. Can someone help? This bug is eleven years old.

... have I gone mad, or have the Debian packages of lilypond 2.24 left out libcairo support? Is this some sort of torture?
Update: I have submitted a Debian bug regarding missing libcairo support in the lilypond 2.24 package.

Should we close this ticket, given we now have wgSVGNativeRendering and related settings merged to master on T208578?

Using the ImageMagick 6.9.11 implementation of encoding AVIF and HEIF (libavif 0.9.3, libheif 1.12.0) is still painfully slow, using a typical large photograph (extracted PNG from a Canon CR3 raw file):

$ file IMG_0652.png
IMG_0652.png: PNG image data, 6984 x 4660, 8-bit/color RGB, non-interlaced
$ time convert IMG_0652.png IMG_0652.avif
real    0m10.608s
user    0m29.370s
sys     0m2.031s
$ time convert IMG_0652.png IMG_0652.heif
real    0m17.438s
user    0m17.694s
sys     0m0.148s
$ time convert IMG_0652.png IMG_0652.jpg
real    0m0.756s
user    0m0.648s
sys     0m0.092s

I agree with @Trougnouf that this issue (use AVIF for thumbnails) shouldn't block progress on being able to upload AVIF files.

@Legoktm @TheDJ - is this likely to happen soon? I'm not sure I can help further other than annoy you by pinging... :-/

Lilypond packages are now built with 2.24 in bookworm and trixie (and sid).

Given the patch on T208578, should we adjust Score's behaviour around serving SVG? So, rather than use srcset (see T134455) to provide SVG with a PNG fallback, we look for the new $wgSVGNativeRendering setting and just serve SVG directly. We'll still need Lilypond to generate PNG, but it is only served for older versions of Mediawiki (using the current srcset behaviour) or where the SVG generated by Lilypond exceeds the new $wgSVGNativeRenderingSizeLimit setting. Another possibility (useful for older versions of Mediawiki) is to detect the presence of the NativeSVGHandler extension and serve SVG directly if enabled. Thoughts?

Fair enough. there seems to be promising recent activity on 1026200 anyway

Surely we can just pull the official upstream binary tarball from GitLab, extract it somewhere, and point LocalSettings.php variables at it? No Debian package required.

You'll have to forgive my total lack of knowledge of Wikipedia internals and toolchains since I'm not an employee, I'm just a Wikipedia editor with an overinflated sense of enthusiasm and can code in PHP, so I offered to help with the Score extension. I'm not sure I'd be much use for most of that?

As it turned out, GIMP added AVIF support to version 2.10.22 in September 2020, see #2668.

I've already done that. How do we get the new version of the Score extension deployed out to Wikipedia so we can enjoy our new SVG output?

This ticket was opened in 2005, nearly twenty years ago. I would like to see T49578 Score SVG output on Wikipedia, hopefully some time before the sun runs out of hydrogen and engulfs our planet in a fiery apocalypse. What can I do to help?

I can reproduce this now... sorry about the noise :)

That's all well and good, in that case, how are we to advance T5593 which has been languishing since 2005? That's nearly twenty years. I would like to see SVG Score output hopefully some time before the sun runs out of hydrogen and engulfs our planet in a fiery apocalypse.

@TheDJ - regarding Score SVG output, T49578 has been merged, but is still open. Not sure what happens now, but this would be a good test for using img srcset, since that's what we implemented in Score.

This was merged over 2 months ago, what happens next? I'd like to get T134455 advanced so we can get on with SVG Score output somehow.

+1 for @Glrx's answer to close; if you want exact PDF-like font rendering, convert it to curves and use parallel hidden text elements if you need the SVG to be indexed in full-text search data. If you want exact rendering and have big walls of text, perhaps you want a PDF file...?

Shall we close this, then?

Don't ask me, I didn't write these PoCs, and I don't know much about Lilypond's use of Scheme, or anything at all about Scheme. I'm just trying to get some PHP code to render music using LilyPond. So does that mean I should see spurious id output in logging?

Debian packaging request for 2.24 in bookworm: 1026200

After a bit of testing, and assuming I should be seeing the system id in the rendered music if this is still vulnerable, I can't reproduce Tim's original PoC or the Notehead hack in Han-Wen's above comment on the following:

Forgive me for wading in from the sidelines with very little dev context as a lowly Wikipedia editor, but I can't reproduce this on 1.39 LTS with Score on the latest master using LilyPond 2.24.
Since the Score extension requires LilyPond 2.24 for supporting SVG output (T49578) which is blocking T181322, T259208, and possibly others, would it be the case that if we upgrade the Score shellbox instances to use LilyPond 2.24 instead, this issue can be resolved? Upstream LilyPond have changed their release process and now issue (64-bit only) binary tarball releases from their GitLab repository directly, which may or may not be helpful.

Update: support for SVG output using LilyPond 2.24 has been added to the master and REL1_39 branches of the Score extension (T49578).

Support for SVG output (with LilyPond 2.24) is now in the Score extension, in REL1_39 and master branches. Not sure what needs to happen before it gets deployed to Wikipedia/media anywhere, though.

Noting here that LilyPond 2.24.0 was released 16 December 2022 - changes. Shall we boldly create an "Update LilyPond in Shellbox container to 2.24" task, similar to the previous one?

Just confirming that I cannot reproduce this (bug?) with LilyPond 2.23.82 and the Score extension from recent master (commit adee410, 14 December 2022) using SVG output.

Hooray :) Now I have a thought about what to do with the existing PNG renders that are being used on pages until they are next edited. Is it possible to create some sort of maintenance task that can process the image store in the background, generating any missing SVG files using the preserved LilyPond file?

Ok, so it turns out we don't even need to get the page size at all, because that was only being used by GhostScript to generate the PNG from the PS output, which we're not doing in this patch. Have pushed an all-in-one commit to Gerrit. I can't see how to easily add a comment to a Gerrit MR, hence the chattiness here (sorry?)

I've pushed two commits ready for review - @TheDJ and @tstarling. Still not sure what the procedure is here; it's been ages since I've had to use Gerrit. Test instance here.
Also: is it worth smooshing them together into one commit?

In T198363#8367491, @valerio.bozzolan wrote:

Also, putting a star on a Wikimedia Commons file would be nice.

I've resolved several things, and pushed an update to Gerrit. I've also found that we don't even need the --ps output and GhostScript any more, because we can fetch the same page size information from the <svg> attributes.

Thanks @TheDJ !

This is still a WIP:

• after the rebase to master, I followed my nose and removed the shell trim code that uses convert, and thus also the ImageMagick dependency altogether.
• I'm still working on paginated output, which is also trimmed correctly in both the SVG and PNG, but it is not currently adding the SVG page images.
• I haven't even looked at the tests yet, so I guess they'll be failing; so much for TDD :)
• I suspect that just using --png instead of deriving them from --ps will also make a bunch more code redundant, and even remove the GhostScript dependency, but I haven't tried that yet.

Ah yes... I remember now. It's been years since I used gerrit... oh, and I need to rebase to master, which will take a while because change

@Aklapper - I get this, and I'm not sure where/who to post or contact:

remote: error: branch refs/heads/generate-svg:
remote: You need 'Create' rights to create new references.
remote: User: jonathanischoice
remote: Contact an administrator to fix the permissions
remote: Processing changes: refs: 1, done    
To ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Score
 ! [remote rejected] generate-svg -> generate-svg (prohibited by Gerrit: not permitted: create)
error: failed to push some refs to 'ssh://gerrit.wikimedia.org:29418/mediawiki/extensions/Score'

I guess I'm only talking about the rendering of an uploaded SVG image, that would be used by [[File:some_image.svg | thumb | etc.]] - which is what $wgSVGConverter is dealing with. It would be up to the uploader to ensure they embedded their fonts or converted their glyphs to paths properly. I'm not sure what impact translation would have. Perhaps we do need to split this task into "support native uploaded SVG image rendering without PNG thumbs" which is basically emulating the behaviour of the NativeSvgHandler extension (I haven't looked at the code to see how it works), and keep this bug as "support embedded SVG in the markup"? Seems to me they are two separate problems, with different concerns applying to each.

Forgive me if I'm posting on the wrong bug, or if this seems naïve, but SVG images (as a src for <img> tags at least) are supported by basically all of the things now (even IE 11 is passable as long as it's not animated):
https://caniuse.com/?search=svg

Here's a 1.35 instance with the 1.35 SVG patch installed:
https://rel1-35.home.jon.geek.nz/wiki/Score_examples

I can't seem to be able to push to Gerrit; in the meantime, the very basic diff is displayed here on GitLab. This should work as long as you have a 2.24 RC version of Lilypond; I have been testing with the 2.23.82 release.

I have a sneaking suspicion that unless we are really still wanting to support really old unsupported browsers, like Internet Explorer 11, we really don't need a PNG fallback. SVG images are supported by basically all of the things, and even IE 11 is passable as long as the SVG is not animated:
https://caniuse.com/?search=svg

Noting here the upstream progress on including the Cairo backend in Lilypond by default: merge request 1610, and lilypond-devel discussion thread.

Is there anything I can do to help this along? I'm a PHP dev with a little time on my hands currently, but it seems as if this is more to do with aligning SVG rendering libs on container OSes? Let me know if I can help in some way. Is this changeset still useful, or has it been superceded by more recent work?