Migrate TiMidity++ to fluidsynth
Closed, ResolvedPublic

Description

T50029: crackling at start of OGG renditions of MIDI files (fixed in TiMidity++ 2.14.0) describes a problem that occurs with the TiMidity++ package currently used by MediaWiki-extensions-Score. @Stemby proposes in T50029#2494744 to switch to fluidsynth.

TiMidity++ does not seem to be maintained very much (https://tracker.debian.org/pkg/timidity) compared to fluidsynth (https://tracker.debian.org/pkg/fluidsynth) while the latter can still perform the MIDI->OGG (or MP3: T181875) conversion.

This task also affects T135597: Move MIDI to audio conversion from Score into TimedMediaHandler as it would no longer be TiMidity++ but the new package.

Ebe123 created this task.Dec 2 2017, 6:40 PM

Sounds good to me at first glance, we should make sure to evaluate the performance of fluidsynth (CPU, memory, wall clock time) versus timidity. Having a more active upstream (and Debian maintainer) all sound great. I tried finding any past CVEs/security issues in fluidsynth but didn't find any, which was a bit surprising to me at least (that said, the only CVEs I could find for timidity were all local crashes/DoS).

divadsn added a subscriber: divadsn.Jan 2 2018, 5:35 PM
Ebe123 removed Ebe123 as the assignee of this task.Jan 2 2018, 7:03 PM
Ebe123 added a project: Google-Code-in-2017.

Removing claim as it's on GCI.

divadsn claimed this task.Jan 3 2018, 11:42 AM

Gonna do some benchmarking then :P

Should I post the results here only or also share the script I used to execute those tests?

Please provide the script so others can reproduce your results.

divadsn added a comment.EditedJan 3 2018, 1:31 PM

@Legoktm alright!

I will be running those tests on my PC running Linux Mint 18.3 with the latest available versions of Timidity and Fluidsynth from the official Ubuntu apt repo.

@Ebe123 you said you can provide me some example MIDIs, would be great if you can ping me later on IRC :)

divadsn added a comment.EditedJan 4 2018, 1:06 AM

Ok, here are my benchmark results: https://docs.google.com/spreadsheets/d/15JEu2IeCM_zJyS3RXDdMHNHoZbxFbKdgJRsEP5LkM3Y/edit?usp=sharing

Conclusion

What I could observe is the fact that Fluidsynth is double times faster while TiMidity++ is lightweight in terms of memory usage. In my opinion both output files sounded good and clear and I think Fluidsynth would improve the page loading speeds at least by half the time.

My testbench:

Acer Chromebook 14 for Work
Intel i3-6100U with 4 GB of RAM
Linux Mint 18.3 (Linux 4.10.0-42-generic)
Used soundfont: FluidR3_GM.sf2

Here is the bash script I made for the tests: https://dpaste.de/fi3w

Follow-up

The sound files generated by TiMidity++ seem to have sometimes the "cracking" issue, while the ones generated by Fluidsynth are clean with no issues. Also I noticed a better sound quality when listening to the generated by Fluidsynth sounds with good headphones.

Ebe123 added a comment.Jan 4 2018, 3:21 AM

Could you upload the resulting audio files for Fluidsynth? Thank you!

@Ebe123 sure, here you go: https://drive.google.com/open?id=1G4fjTAy7yF280kfAHMeEe7VIvVyPrMpx

(I actually couldn't upload them here without getting an error)

One comment regarding some sound distortions made by TiMidity, they are present in my files above. At the beginning of the track you can hear a "crack", while Fluidsynth doesn't have that issue.

Change 402148 had a related patch set uploaded (by Divadsn; owner: Divadsn):
[mediawiki/extensions/Score@master] Migrate TiMidity to fluidsynth

https://gerrit.wikimedia.org/r/402148

divadsn added a comment.EditedJan 5 2018, 11:00 AM

@Legoktm great, so should we deprecate TiMidity++ soon as fallback when the patch gets merged?

Or wait until Fluidsynth has been tested carefully for production, @Ebe123?

TheDJ added a comment.Jan 5 2018, 12:04 PM

@divadsn regardless of what is done, before fluidsynth can be put in production, there will have to be a security review by the security team, please keep that in mind.

@TheDJ sure, I am aware of that too. Until then I will work on T181875.

Change 402148 merged by jenkins-bot:
[mediawiki/extensions/Score@master] Migrate TiMidity to fluidsynth

https://gerrit.wikimedia.org/r/402148

divadsn closed this task as Resolved.Jan 10 2018, 1:10 PM

Now it's possible to slowly migrate from TiMidity++ to Fluidsynth and then after a period of time drop support for TiMidity++ :)

TheDJ awarded a token.Jan 21 2018, 5:19 PM