{"id":2399,"date":"2019-03-24T18:14:26","date_gmt":"2019-03-24T22:14:26","guid":{"rendered":"http:\/\/wizardwalk.com\/newblather\/?p=2399"},"modified":"2019-03-24T18:14:26","modified_gmt":"2019-03-24T22:14:26","slug":"music-editor-developers-log-soundfont-insanity","status":"publish","type":"post","link":"https:\/\/wizardwalk.com\/newblather\/?p=2399","title":{"rendered":"Music Editor Developer’s Log: Soundfont Insanity"},"content":{"rendered":"

For the past week, I’ve been trying to give my music editor<\/span>1<\/sup><\/a><\/span> the power of sound. I looked into the new Web MIDI API standards, but those are more for sending and receiving MIDI messages, not playing sound, so that’s no help. (Though it may be something to look into later for other features, of course.)<\/p>\n

So instead I’ve been looking into the Web Audio API, which does the trick, and has mostly what I need. Actually, it has everything I\u00a0need<\/em>, but not everything I\u00a0want<\/em>. I want the sounds to sound as good as possible, which means the instrument samples must loop for sustains (as a MIDI synth would).<\/p>\n

First I experimented with MIDI.js<\/a>‘s implementation of sample playing. With pre-rendered soundfonts<\/a>, I could easily play samples for all the basic MIDI instruments. Problem with this implementation is that the instruments don’t loop! (Or at the very least, they don’t seem to read in the looping data saved in the soundfont.) Instruments such as strings, which can sustain indefinitely, really deserve some decent looping.<\/span>2<\/sup><\/a><\/span><\/p>\n

So I moved on to experimenting with a library called sf2synth.js<\/a>. I can’t understand the Japanese comments (the developer seems to be from Tokyo), but this implementation seems to load in soundfont files much more completely, and actually reads in and uses the looping data! Woohoo!<\/p>\n

But even it has a problem. When I play a note from the Musyng Kite soundfont<\/a>\u00a0 (which is the soundfont I’m currently using for experimental purposes) in the Polyphone Soundfont Editor<\/a> (which is a great piece of software), it sounds great. But when it’s played back in the browser through sf2synth.js, it sounds more bland.<\/p>\n

Here is what I think is happening…<\/p>\n

If we look at a preset in Polyphone, we can see that it’s actually made up of multiple instruments; below you can see that “Strings Ensemble” is actually made up of 8 layers.<\/p>\n

\"\"<\/a><\/p>\n

To me, it sounds like sf2synth.js is only playing\u00a0one<\/em> of these layers, instead of all of them like a true soundfont player should.<\/p>\n

So my mission for this week is to dig into the sf2synth.js code, try to understand how it’s loading and playing sounds from the soundfont file, and try to give it the ability to play all the layers in a preset that it should. Polyphone is open-source, so I can also dig around their code to see how they’re loading in and parsing \/ interpreting sf2 files.<\/p>\n

I probably only want to spend two weeks max on this; if I can’t figure it out after two, I’ll just have to settle for suboptimal sounds and move on. I can always come back to soundfont programming later. It’s more important to get a working prototype finished by the end of July. 129 days left!<\/p>\n","protected":false},"excerpt":{"rendered":"

For the past week, I’ve been trying to give my music editor the power of sound. I looked into the new Web MIDI API standards, but those are more for sending and receiving MIDI messages, not playing sound, so that’s no help. (Though it may be something to look into […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spay_email":"","jetpack_publicize_message":"","jetpack_is_tweetstorm":false,"jetpack_publicize_feature_enabled":true},"categories":[19],"tags":[100,791],"jetpack_publicize_connections":[],"jetpack_featured_media_url":"","jetpack_shortlink":"https:\/\/wp.me\/p7gI4B-CH","_links":{"self":[{"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/posts\/2399"}],"collection":[{"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2399"}],"version-history":[{"count":3,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/posts\/2399\/revisions"}],"predecessor-version":[{"id":2406,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=\/wp\/v2\/posts\/2399\/revisions\/2406"}],"wp:attachment":[{"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2399"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2399"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wizardwalk.com\/newblather\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2399"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}