I have begun an RfC on Commons for this.

The RfC has ended without a formal closure, and is now archived. There was majority support for all six file types that were options by the end of the RfC, although five users opposed allowing MuseScore uploads mainly due to issues with the file format.

I asked for assistance on how to proceed at the administrators' noticeboard, but no one has responded. If that gets archived then I'll probably just make a commit to mediawiki-config to add all of the file types and see what happens.

In T208494#4885398, @Jc86035 wrote:

The RfC has ended without a formal closure, and is now archived. There was majority support for all six file types that were options by the end of the RfC, although five users opposed allowing MuseScore uploads mainly due to issues with the file format.

I'm not sure adding 5 different file types for the same type of content is the best idea, but okay...

I asked for assistance on how to proceed at the administrators' noticeboard, but no one has responded. If that gets archived then I'll probably just make a commit to mediawiki-config to add all of the file types and see what happens.

Someone will probably need to add support for each file type in MediaWiki first, possibly in the Score extension. @Ebe123 or @TheDJ might be able to advise on how to proceed.

Also, given that these formats all seem to be text based, is there a reason we should be treating them as files, and not as editable wiki pages?

A while ago I wrote https://www.mediawiki.org/wiki/Manual:Adding_support_for_new_filetypes specifically so that I dont have to repeat all this every single time someone asks for a fileformat ;)

For communication/tracking, I suggest filing separate subtasks for each specific format. Each ticket can answer all the questions in a list in the description of the task.

And then find developers. Which is probably (as always) the biggest problem to solve.

Also, given that these formats all seem to be text based, is there a reason we should be treating them as files, and not as editable wiki pages?

As far as I know, there is no support in any part of the media engine to use revisions table instead of the image table.

In T208494#4885952, @Legoktm wrote:

I'm not sure adding 5 different file types for the same type of content is the best idea, but okay...

This is mainly because all of the formats are slightly incompatible with each other when importing/exporting, and most users are unlikely to prefer using more than one. We already host both OGGs and MP3s, as well as both TIFFs and PNGs.

Someone will probably need to add support for each file type in MediaWiki first, possibly in the Score extension.

Not necessarily. To my knowledge MP3s couldn’t be played for years until support was added in the Score extension, so it’s not a deal breaker (for me, at least) if it’s not possible to render or play any of the files.

text based

SVG is XML, but SVG files can still be uploaded to Commons. I also doubt that it would be possible to textually edit compressed MuseScore/MusicXML files.

I've removed site-requests tag until this is completely actionable.

@TheDJ, I've added a checklist at the top. I am not aware of anything that isn't already posted here and in the RfC, so it should be possible for others to add checklists for the subtasks if desired.

The interactive concerns would probably only be relevant if anything is implemented with Verovio; I anticipate using MuseScore's existing export functions would be much easier than adding interactive playback for it.

What does "Your thumbnail (esp. when interactive) should be usable [...] for screenreader users" mean?

@Jc3s5h thank you for taking the steps, it is very valuable. As you note, adding thumb nailing support is a lot more steps than a simple upload ;)

"Your thumbnail (esp. when interactive) should be usable [...] for screenreader users" mean?

It means that anything you render in a wiki page should be accessible and understandable for screenreader users. At the very minimum that it shouldn't be obtrusive, but especially when you have web players etc.

In T208494#4891702, @TheDJ wrote:

@Jc3s5h

...Wait, you were replying to me, right?

sigh auto complete ;)

Anyway. Thanks for clarifying that.

On enwiki yesterday I notified HLHJ on their talk page about this, since they don't have a Phabricator account. As noted there, I'm not actually sure whether or not it's necessary to enable thumbnailing before enabling uploads. I don't think it is, since (as I noted in the RfC) MIDI still can't be played through the normal file syntax and has to be passed through an empty <score/>, giving precedent for enabling uploads (2005?) before enabling playback (after 2011). There was also no expectation that thumbnailing would certainly be possible, as far as I'm aware.

@Jc86035 wrote:

It may be desirable to integrate musical notation files into MediaWiki, most likely by allowing their display as thumbnails and possibly allowing audio playback from them. In November I made a Community Wishlist Survey proposal to propose the technical development of such capability. Even though the proposal has not succeeded, it may still be advantageous to host these files on Commons without using them directly in other projects.

Furthermore, allowing uploads first would not necessarily hinder display in pages, since LilyPond and MuseScore can export to PNG/PDF/OGG, and those could be used in lieu of working thumbnails. Once there is sufficient support then those could be replaced with something like [[File:LilyPond.ly|thumb|300px|page=1|vorbis=1]].

I take it that by "thumbnail", we mean visual display, and "playback" aural display.

I participated in the RfC discussion. I think there were a few unresolved issues there, and I'm sorry it lapsed; my fault too.

One big one: Plaine & Easie Code is used as a standard in library catalogues. There are over a million music incipits available under CC-BY in this format, which seems a decent argument for inclusion.

Perhaps an even bigger one; as @Legoktm said, this is a long list of formats, and it might be good to prioritize them. Here is my unilateral attempt, based in part on a severly underinformed assessment of how hard each thing will be. Comment and crit welcome:

1. Just uploading LilyPond (and ABC) would give a lot of functionality, even with no display ability. I'd favour being able to edit them online as markup, but no idea how hard that would be. It would be great to be able to transclude them into, say, Wikipedia articles, but even cut-and-paste would be usable.

2. MusicXML upload, and MusicXML to Lilypond conversion would be a good next step. There exists copyleft software to do it. MusicXML is an established format, a W3C recommendation, and the most widely used interchange format. It had the broadest support at the RfC on music formats.

3. Braille music. Gently disagreeing with part of @Jc86035's much earlier comment; there is also a visual web display for the Braille format, and it is really useful if one is sighted. Braille integration seems to be a discrete task, and technically looks as though it could be fairly easy, with a helpful dev who has already written all the (GPL) software components and is interested in integrating them. Details on subtask. There are also conversion utils, so this can be seen as an accessibility interface, not just a format.

4. Integration of the Verovio score renderer (LGPLv3 license). It will render (sight and sound, example) essentially everything proposed, except Lilypond (done), Braille, and MuseScore (whose inclusion I opposed). Verovio can visualize MusicXML (used by everyone) , MEI (a format used extensively by academics for PD works), Plaine & Easie (librarians in catalogues), ABC, and **kern (still figuring that one out). Display-less uploading of these formats could, of course, be done earlier.

I feel as if some of the other RfC participants might have more useful contributions here...

there is also a visual web display for the Braille format, and it is really useful if one is sighted

I was presuming that those who are sighted would prefer to use the other formats. I've never used Braille music, so I'm probably wrong here. I think if the dev is willing to implement it then that's great.

I still don't think it's fair to exclude MuseScore, if only for its larger user base and audience (and the fact that it's still open source). Combining e.g. MuseScore's soundfonts and/or audio generation with Verovio might also be useful, since as far as I can tell Verovio only has one lonely piano soundfont.

I agree that LilyPond would have to be prioritized, since a MediaWiki extension already exists for it. I don't really mind how the formats are prioritized, although I would put MuseScore either before or after MusicXML.

Editing them as markup could be possible from the beginning, since there is a user script on Commons for editing SVGs from the browser window which could be repurposed for LilyPond/ABC. The check to abort the script if the file on the page is not an SVG is one line long, so it would be fairly easy to make it into a LilyPond editor (although the preview function would have to be omitted, since it's SVG-specific). However, I don't think it should be a priority.

I was presuming that those who are sighted would prefer to use the other formats. I've never used Braille music, so I'm probably wrong here. I think if the dev is willing to implement it then that's great.

I believe that sighted transcribers are the main users.

(Is it OK for me to comment here? If not, please let me know)

I think it's important to point out that "MuseScore" files are not really a recognized "open" format. Yes, they are used as the output of an open-source notation engraver, but they are primarily designed as an internal format for the software. AFAIK, there is no guarantee of versioned stability of MuseScore files between releases, so even minor updates to the software could see major changes to the internal format. See:

https://musescore.org/en/node/41106#comment-181251
https://musescore.org/en/node/13837

@Ahankins, yes, you're welcome to comment here.

According to that comment by Werner Schweer at https://musescore.org/en/node/41106#comment-181251, shouldn't LilyPond have similar problems (although perhaps less troublesome) with breaking changes? The MediaWiki extension for LilyPond has to use the actual LilyPond interpreter anyway, so presumably for any MuseScore extension the same would be done with the MuseScore source code (unless the files are converted to a different format first).

I agree it's important to point it out, but it's not really fair to single out MuseScore for using an "internal" format when the only notation software currently supported by MediaWiki could be described accurately by almost the same words.

Yes, Lilypond does have similar problems. I personally wouldn't recommend it as a stable format either, but since it is already supported on MediaWiki I didn't see the point of arguing that it should be removed.

In general, "good" notation formats should be independent of any particular software. I could write a fairly long post on the thinking behind why this should be the case (and would be happy to do so if you think it useful) but the gist of it would be that music notation is complicated, and software implementations typically resort to 'hacks' to get things to display correctly, at the expense of accurately encoding and 'representing' the underlying music.

T49528, "Create a VisualEditor plugin tool to add/edit musical scores", was also discussed in the RfC, and should probably be referenced here.

Note that there was an overwhelming support for allowing these file types during the proposal discussion (request for comment).

In T208489#4928059, @Jc86035 wrote:

In T208489#4927904, @Lydia_Pintscher wrote:

In T208489#4927702, @Jc86035 wrote:

• Is there a good reason to favour LilyPond over other music notation formats? [...]

This is the one that can be reused inside the Wikimedia projects and that I have seen outside it as well. Any new one would need a lot of additional development time to support it that I'm currently not really convinced is worth it.

[...]
On the other hand, if it's exceedingly unlikely that anyone will put in the work to create new extensions, then the scope of T208494 should probably be considerably reduced to only allowing Commons uploads of the formats, since thumbnail generation would effectively require four new extensions (as well as work to enable thumbnails for LilyPond files). If that occurs, it would probably affect this task, since it would become possible to upload music notation files to Commons much sooner. It would be possible to instead indefinitely postpone enabling all formats other than LilyPond on Commons, but this would go against the consensus of the Commons RfC.

@TheDJ It's still not clear to me whether there is a need for the thumbnail generation to be enabled before uploads are enabled, and how the need for such is to be determined. Could you clarify this part of the procedure?

If it's determined that there is no need for thumbnails, then the amount of work which has to be done before enabling uploads is significantly reduced. If it's determined that there is a need for thumbnails, then a very likely outcome (as far as I'm aware) is that only LilyPond uploads are eventually enabled, and support for the other formats remains in development hell indefinitely.

If thumbnails aren't generated at first, then the main effect would be that users would be inconvenienced by having to export the files to PDF before being able to use them, and uses of files would need to be replaced upon enabling thumbnails for each format. On the other hand, even this would be an improvement over the current situation (i.e. scores have to be shared either as wikitext or through other websites); and the fact that exporting to PDF is possible for all of the formats – and would probably be done anyway in the MediaWiki thumbnail generation – means that thumbnails are not a necessity for the files to be usable. As such, I think it would be more practical to enable uploads first, although (again) I don't know how the decision is to be made.

In T208494#4894920, @Ahankins wrote:

AFAIK, there is no guarantee of versioned stability of MuseScore files between releases, so even minor updates to the software could see major changes to the internal format.

There is guaranteed stability in minor and patch releases. This means that the format is rarely changed, and when it is, the new features must be introduced in a backwards-compatible way so that the file can still be opened in older releases (but the new features will be ignored). An example of this happening is when support for "cresc." and "dim." lines was added. This was done in such a way that the lines would be rendered as hairpins in older versions.

Backwards-incompatible changes must only occur when the major version is increased. There is an argument to say that there should never be backwards-incompatible changes, but I would argue that these changes are necessary to allow the software to improve. This is in contrast to MusicXML which goes to great lengths to preserve backwards compatibility, at the cost of preserving some archaic syntax that it would be much better to get rid of. (An example of this is measure-repeats, which must contain both the instruction to repeat the previous measure, as well as the actual content of that measure written out all over again.)

In T208494#4894933, @Ahankins wrote:

In general, "good" notation formats should be independent of any particular software. I could write a fairly long post on the thinking behind why this should be the case (and would be happy to do so if you think it useful) but the gist of it would be that music notation is complicated, and software implementations typically resort to 'hacks' to get things to display correctly, at the expense of accurately encoding and 'representing' the underlying music.

True, but unfortunately notation programs also use 'hacks' to export to non-native formats. It's a choice of using a native format (like MuseScore format) and getting optimum results in that one program (and sub-optimum results in everything else), or using a non-native format (like MusicXML) and getting sub-optimum results in all programs.

In T208494#4895325, @HLHJ wrote:

T49528, "Create a VisualEditor plugin tool to add/edit musical scores", was also discussed in the RfC, and should probably be referenced here.

I would love to see this, but I can more or less guarantee it won't happen, or if it does it will be extremely limited in functionality for the foreseeable future. There has been talk about MuseScore creating an in-browser editor, but the talk is always about ways of taking the existing desktop code and running it in the browser, not starting from scratch in Javascript because we know that it is not a realistic proposition. LibreOffice has an online version called Collabora, but again this is just running the desktop program on a server, not a client-side implementation like Google Docs, and sheet music is much more difficult than text.

My personal preference would be for Wikimedia to allow uploading of some or all of these formats, but insist that files are only editable in the format that they were originally uploaded in. There could be an option to download in other formats, but those files would be generated automatically from the original file.

However, there should be some thought given to the fact that there are already other projects creating these files, and that it would probably be better to avoid duplicating effort where possible. Perhaps where scores already exist in another project, Wikimedia should simply link to the other project's file, perhaps storing it's own copy as a backup/mirror in case the other project goes down?

In T208494#4885952, @Legoktm wrote:

In T208494#4885398, @Jc86035 wrote:

The RfC has ended without a formal closure, and is now archived. There was majority support for all six file types that were options by the end of the RfC, although five users opposed allowing MuseScore uploads mainly due to issues with the file format.

I'm not sure adding 5 different file types for the same type of content is the best idea, but okay...

I have to be honest, I'm confused by this: what harm is there in allowing (e.g.) PNG and GIF and TIFF and JPEG all at once? Yes, those formats have relative strengths and weaknesses but they are all (minus animated GIFs), static images. It's okay to have PDF and DJVU. I just don't see what is *worse* about including a broader array of free file formats. If one is clearly superior (e.g. how SVG is a better format for many static images than any raster option), then won't users generally gravitate toward that? Plus, it's sometimes cumbersome to convert files, so it's a good thing to allow an easy way to upload (e.g.) MP3s rather than require OGG conversion.

In T208494#4930166, @Koavf wrote:

In T208494#4885952, @Legoktm wrote:

In T208494#4885398, @Jc86035 wrote:

The RfC has ended without a formal closure, and is now archived. There was majority support for all six file types that were options by the end of the RfC, although five users opposed allowing MuseScore uploads mainly due to issues with the file format.

I'm not sure adding 5 different file types for the same type of content is the best idea, but okay...

I have to be honest, I'm confused by this: what harm is there in allowing (e.g.) PNG and GIF and TIFF and JPEG all at once?

Each one will need its own work to be integrated into MediaWiki. That means, we'll need to add its own security screening, its own validation, its own rendering, its own playback, its own set of tests, its own interface messages, its own server-side dependency management, and so on. Adding each format costs maybe US$5–10k of staff time, plus maybe $1–2k per year of on-going security issues / fix-ups / testing / hardware growth requirements / etc. (some formats, like SVG of GIF, cost more – because they're less secure, have complex pipeline issues, or other problems). As each of these formats is novel and rarely used (in comparison to PNG), they'll be towards the upper end of these values.

Adding all seven formats at once probably would save some effort, but we're still talking a significant sum to spend on a relatively low-interest niche, which would instead be spent on other things.

In T208494#4930166, @Koavf wrote:

In T208494#4885952, @Legoktm wrote:

I'm not sure adding 5 different file types for the same type of content is the best idea, but okay...

I have to be honest, I'm confused by this: what harm is there in allowing (e.g.) PNG and GIF and TIFF and JPEG all at once? Yes, those formats have relative strengths and weaknesses but they are all (minus animated GIFs), static images. It's okay to have PDF and DJVU. I just don't see what is *worse* about including a broader array of free file formats. If one is clearly superior (e.g. how SVG is a better format for many static images than any raster option), then won't users generally gravitate toward that? Plus, it's sometimes cumbersome to convert files, so it's a good thing to allow an easy way to upload (e.g.) MP3s rather than require OGG conversion.

The cost is that each format needs to have code written for it, plus maintained. We also need to review different external software for each one, and test it during OS upgrades.

PNG is good for some files, GIF is for animated stuff, TIFF is good for bigger files, and JPG is uh, the format that most consumer cameras use? I'm not really an expert tbh, just my general worries about maintenance costs. That said, if people are willing to put in the development effort and help with maintenance and other people who are more experienced in this area think it's a good idea, I'm happy to help however I can.

It seems to me that SVG is a good candidate. It is AFAIK the only vector graphic format supported by decently recent browsers. Thumbnails (PNG) can be easily be generated from it if necessary, including client-side with JavaScript and Canvas.

In T208494#4930646, @lpugin wrote:

It seems to me that SVG is a good candidate. It is AFAIK the only vector graphic format supported by decently recent browsers. Thumbnails (PNG) can be easily be generated from it if necessary, including client-side with JavaScript and Canvas.

I assumed it would be PDF due to some/most of the formats having page layout settings (and rendering multi-page scores as one thumbnail would probably be undesirable); MediaWiki can render single pages of PDFs as thumbnails. I don't really know how this would work.

Notably, the Score extension currently doesn't use SVG because of librsvg font issues (see T49578).

Notably, the Score extension currently doesn't use SVG because of librsvg font issues (see T49578).

OK, this is good to know. So SVG would be fine as long as librsvg is not used in the pipe to produce them.

@Jc86035: No, librsvg is not really a blocker. It's that the option for SVG layout in Lilypond is in the next version. For PNG fallback, imagemagick could still be used (instead of librsvg)

In T208494#4928580, @Jc86035 wrote:

@TheDJ It's still not clear to me whether there is a need for the thumbnail generation to be enabled before uploads are enabled, and how the need for such is to be determined. Could you clarify this part of the procedure?

Thumbnails, or rather ANY form of transforming the raw file into output other than a browser download of that exact original file is not required, but of course severely limits the usability of such files within a wiki. It basically only allows preservation of the file.

A minor lyrical winkle. If we could store scores on Commons, we could transclude them into other wikis as we do images, and avoid repeatedly re-transcribing them. But lyrics would be best stored and transcluded seperately, because it's common for one score to have multiple sets of lyrics.

Example: Tochter Zion, the other setting (with minicule melodic variations, often now also used for Tochter Zion) See, the Conqu'ring Hero Comes, "À toi la gloire O Ressuscité" and its translation Thine be the Glory, etc.. These have multiple score transcriptions on multiple languages of wiki.