melody project

TuneSage progress update 5

It’s been over a year since my last TuneSage update, but work has been progressing. Work on the backend was slow and challenging; I spent some time going down quite a few dead-ends. But it’s actually generating melodies now, so I should be able to release some output samples sometime soon, fingers crossed.

I signed up for this year’s Y Combinator’s Startup School once again. I did it once back in 2019, but other than making a landing page (TuneSage.com), my progress was sparse. I just needed a lot more time than I imagined to make progress with the backend.

Startup School’s Course Guide says: “If you haven’t launched yet, make it a goal to launch during the program and get your first users!”

OK, I guess I’ll make it a goal then! Granted, I have repeatedly failed at making enough progress to launch whenever I have set it as a goal, but one must keep trying I suppose.

My question is: how many features do I need to launch with? For better or worse, my current plan is to just launch the product as a basic melody generator to start with.

So what do I need to do to launch?

  • Prepare the backend
    • Train the AI on more melodies (using public domain melodies)
    • Generate melodies in a variety of styles (these will be basic to start with)
  • Add at least some simple chordal accompinement features to frontend
    • e.g. root notes, arpeggio patterns, alberti bass, etc.
  • Overhaul frontend design (lots of tedious web design) and finalize
  • Figure out what soundfonts TuneSage will use
  • Figure out deployment and version control (honestly this can probably wait until after launch, but not too long after)
  • Create user account system
    • Create new account
    • Confirm email (if necessary)
    • Log in / out / reset password
    • Edit optional personal info
    • Usage stats
    • Terms and conditions
  • Register company
  • Find some payment system to use
  • Allow for a trial period (and decide exactly what that consists of)
  • Install some analytics system (so I can keep track of user engagement or whatever)
  • Stress testing? (In my experiments with “trovedex”, the database kept going down; I really don’t want that to be a problem. Anyway, I can always do invite-only if the system is too stressed, but that would be a good problem to have)
  • Launch! Update front page with information

Is that it? Am I forgetting anything?

Of course, there are many more features I’d still like to add; AI can be used for a lot more than just generating melodies. But it’s a starting point, and melodies are the one area I think other AI music systems struggle with the most at the moment.

So how long will all that take? Can I finish it in a few weeks? Startup School lasts for 7 weeks, so if I can do it in 3 or 4 weeks, that would be awesome. Considering how long things have taken me in the past, however, it will probably take me… 12 years. But for now let’s daydream:

  • Week 1: Finish backend and overhaul frontend
  • Week 2: Soundfont and user account system, start releasing samples
  • Week 3: Register company, install payment and analyctics systems
  • Week 4: Set up trial, stress testing, front page update, and launch!

That’s probably wishful thinking, but it’s better than nothing.

By S P Hannifin, ago

Music Editor Developer’s Log: More Cowbell

I got the soundfont to work, or at least to work well enough for my prototype-creating purposes. It will need some fine-tuning in the future, but if I can manage to actually turn this software into a business, it would be nice to create a custom soundfont for it anyway.

I’m now almost to the point where I can start using this software to actually write some music, but I’ve still got a number of controls and GUI elements (buttons and stuff) to program. I need to add the abilities to do mainly the following:

  • add and delete measures
  • edit note / track variables such as
    • release time (how long it takes an instrument to fade away after it has stopped playing)
    • volume / velocity
    • stereo position (left or right)
  • edit reverb settings
  • save and load files
  • export and load MIDI files (depending on time; this feature isn’t too important yet)
  • export MP3 or WAV files (at least look into it; if this is too time-consuming, it’ll be something to look into in the future)

I think that’s mostly it. And none of that stuff (save for perhaps the last one) should be too terribly difficult to code. So I think I can get it done this week!

After that, I will probably be a bit more secretive as I begin adding the “secret ingredients” which are my amazing world-class AI music generating algorithms, which will be the secret sauce of the business. For that, I will probably have to buy a dedicated server (or VPS), as those algorithms will be executed server-side. That’ll be fun.

Hopefully I’ll also be able to use this editor to actually compose some new tracks this April. I owe my few Patreon subscribers probably around a dozen or so tracks, and I want to get that new album out, which just needs one or two more tracks. And it would just be a good test of the software, even without the AI features, to see what composing with it is like. 122 days left!

Oh, what exactly will constitute success come July 31st? I mentioned earlier that success will mean that the software will either be at a point in which it’s ready (or close-to-ready) to actually market and sell, or in which a working prototype is ready to show to investors. Of course, those possibilities are not mutually exclusive, but at least one must be the case. But what does the latter mean? What will make it “ready” to show?

Anything really, so I can’t lose!

Seriously, though, it will mean that the software should be able to auto-write a complete song (minus lyrics) on its own. That’s melody, chords, orchestration. The algorithms are done, it’s just a matter of making them usable to an end-user and making their output as good as possible.

I’d ideally like the software to be able to compose something with the complexity of a Mozart symphony. That would be the true peak of Parnassus. And I’m positive we’ll have that soon enough. Maybe not by July 31st, but it would certainly be awesome, no?

By S P Hannifin, ago

Music Editor Developer’s Log: Soundfont Insanity

For the past week, I’ve been trying to give my music editor1 the power of sound. I looked into the new Web MIDI API standards, but those are more for sending and receiving MIDI messages, not playing sound, so that’s no help. (Though it may be something to look into later for other features, of course.)

So instead I’ve been looking into the Web Audio API, which does the trick, and has mostly what I need. Actually, it has everything I need, but not everything I want. I want the sounds to sound as good as possible, which means the instrument samples must loop for sustains (as a MIDI synth would).

First I experimented with MIDI.js‘s implementation of sample playing. With pre-rendered soundfonts, I could easily play samples for all the basic MIDI instruments. Problem with this implementation is that the instruments don’t loop! (Or at the very least, they don’t seem to read in the looping data saved in the soundfont.) Instruments such as strings, which can sustain indefinitely, really deserve some decent looping.2

So I moved on to experimenting with a library called sf2synth.js. I can’t understand the Japanese comments (the developer seems to be from Tokyo), but this implementation seems to load in soundfont files much more completely, and actually reads in and uses the looping data! Woohoo!

But even it has a problem. When I play a note from the Musyng Kite soundfont  (which is the soundfont I’m currently using for experimental purposes) in the Polyphone Soundfont Editor (which is a great piece of software), it sounds great. But when it’s played back in the browser through sf2synth.js, it sounds more bland.

Here is what I think is happening…

If we look at a preset in Polyphone, we can see that it’s actually made up of multiple instruments; below you can see that “Strings Ensemble” is actually made up of 8 layers.

To me, it sounds like sf2synth.js is only playing one of these layers, instead of all of them like a true soundfont player should.

So my mission for this week is to dig into the sf2synth.js code, try to understand how it’s loading and playing sounds from the soundfont file, and try to give it the ability to play all the layers in a preset that it should. Polyphone is open-source, so I can also dig around their code to see how they’re loading in and parsing / interpreting sf2 files.

I probably only want to spend two weeks max on this; if I can’t figure it out after two, I’ll just have to settle for suboptimal sounds and move on. I can always come back to soundfont programming later. It’s more important to get a working prototype finished by the end of July. 129 days left!

By S P Hannifin, ago

Y Combinator

Y Combinator looks like something I’d really like to do. Looks like it would be perfect for the symphony generator app I’ve been working on. As they say on their site:

Twice a year we invest a small amount of money ($120k) in a large number of startups.

The startups move to Silicon Valley for 3 months, during which we work intensively with them to get the company into the best possible shape and refine their pitch to investors. Each cycle culminates in Demo Day, when the startups present their companies to a carefully selected, invite-only audience.

The deadline for their upcoming Winter 2019 cycle is October 2nd, less than two months away. Unfortunately I’m not so sure I can finish a working prototype by then. Worth a try though. Might have to wait until their next cycle after that. Regardless, this looks like something to aim for!

By S P Hannifin, ago

More new music and what happened to the melody generator

I uploaded a new piece of music to YouTube earlier today called The Stormbringer’s Apprentice:

I write in the description:

This piece provides the themes for a villain called Stormov, from my book Insane Fantasy. He’s an apprentice for the mysterious “Stormbringer”, and helps to entice new recruits for his master’s evil plans. His themes are mostly dark, but there is a hint of adventure in there as well, as he lures potential helpers with temptations of worldly power.

Also, this is my first video to feature my own attempt at programming my own custom “music animation machine” programmed in Java with jMonkeyEngine. The frame rate is not as smooth as I’d like it to be (I’m still screen-capturing it), but it’s not horrible at least. Anyway, I think I’ll enjoy playing around with it. Obviously it’s inspired by Stephen Malinowski’s work, as I’ve been using his decade-old “Music Animation Machine” program for a long time now.

As mentioned above, this piece is animated with my own custom “music animation machine.” I may release the source code for it at some point for anyone who’s interested in it, but you’d have to download jMonkeyEngine to compile it and run it yourself; I’m not really interested in making into a standalone program right now, as I hope to continue fooling around with the code. The code is also sloppy and contains features I never finished programming, as it’s all part of an ongoing MIDI sequencer project. Also, it doesn’t actually even play MIDI files… it only makes the shapes and animations based on a MIDI file, which is all I need it to do as I sync videos with tracks recorded from Overture in Windows Movie Maker before adding titles and uploading to YouTube.

In case you missed it, last week I posted this track called Storybook Overture:

So that makes five tracks completed so for my upcoming album (which I have no idea what I’ll name yet): Lullaby of the Westwind Woods, The Storm Cometh, A Stargazer’s Lullaby, Storybook Overture, and The Stormbringer’s Apprentice. Together they amount to over 20 minutes, so I’m over 1/3rd finished the album!

In other news, I’ve had several people email me over the last few months asking about whatever happened to the melody generator. Well, I was contacted by a shady underground group that secretly controls the world, and they warned me that the world simply is not ready for something so powerful.

Actually, the project just got to be too frustrating. It was getting enough interest that people were thirsty to learn how it worked, yet not enough interest to fund a Kickstarter, and I just didn’t (and don’t) have the time to give it the attention it needs. I of course haven’t given up on it, but it’s on the back-burner for now until I can get my life sorted a bit more. In the meantime, I’d just rather spend my free time writing music and books and programming games, as those projects tend to actually generate some income. Not much, but something. So I have absolutely no idea when I’ll be able to get back to working on the melody generator. Could be later this year, could be two or three years down the road.

I have an email list here if you’d like to be updated when I actually return to the project: Melody Generator News.

By S P Hannifin, ago

Trying a Kickstarter sometime soon…

I’ve been in a bleak creative rut lately. I feel like doing nothing creative at all. I’ve had no inspiration, interest, or will to do anything. Go to work, come home, eat, and waste time watching movies and surfing the web. A dull, boring, purposeless, vacuous existence.

So in an effort to give myself a bit of purpose, I’ve been working on putting together a Kickstarter campaign called “Give me some money to smile again.”

OK, it’s not called that… it’s actually fundraising for that book on melody that I’ve been writing on and off for years. The Secrets of Melody. It details my theory of melody, including how my melody generators work. Basically, if I can raise the funds, I can finish it by March 2016. Of course, I have hardly any online following, so I know it’s a long shot. Seems worth a try though, especially as I feel no great desire to do much else at the moment.

And if I fail to raise the funds, I’m going to go ahead buy an Oculus Rift dev kit and start having some fun with it to make myself feel better. So it’s a win-win for me. And I might buy one regardless because I’ve been dreaming of VR and I’ve got some fun ideas for projects I’d like to try with it. But if there’s enough interest for my book on melody, I’d love to make that a priority.

Anyway, I just have to make the Kickstarter video, which is hard because I’m the self-conscious type who hates looking at a video of himself. But I’ll try anyway.

So look out for that sometime this week.

By S P Hannifin, ago

February happenings…

Haven’t gotten much done this week. I’m adjusting to a slightly new work schedule with a bit more hours (which is good for the paycheck), plus some chest congestion has been keeping me from getting to sleep, and I keep waking up with headaches. So I haven’t been very good at finding large chunks of time to do any very focused work.

Anyway, I’ve been a bit torn lately between competing interests: should I work on my fiction writing, or my music software? Whenever I commit to one, I feel the pull of the other. Ultimately, though, I think I’m going to focus on programming my music software for a while. I’m excited about its potential, plus it would be awesome if I could turn it into a business and be self-employed with it. Not that I’m not arrogant enough to believe I could do the same with writing with more work and patience, but music software that’s like nothing else on the market (as far as I know) probably stands a better chance.

By S P Hannifin, ago

There’s too much stuff I want to do…

I recently finished a few weeks of freelance programming, creating some custom software that allows the client to quickly format / generate bid documents for a demolition company.  It’s nice to finish something that’s actually productive, something I haven’t done much of this year…

Anyway, I’m now back to having a bunch of free time and a continuing burning desire to not have to get a real job that will steal it all away again (even though I’m really broke). I have a lot of projects I’d like work on, including:

  • Finish writing that book on melodies that I’ve been planning for years
  • Write a fantasy series and indie-publish it on Amazon Kindle, Smashwords, etc.
  • Write more music and put together another album
  • Write and draw a web comic based on that cartoon series I created back in 2012
  • Write fairy tales — I’ve been wanting to write short weekly fairy tales, just for fun
  • Do something with computer game programming — I have a short mystery game I planned out a year ago that I still haven’t done anything with.  I’d like to learn how to use Unity or something, create some pixel art, and actually try to make it

Of course, the problem is that making any sustainable revenue from any of these areas of interest takes time… too much time. So much time that a sane person would get a real job in the mean time.

But I really don’t want to have to be sane.

The other problem is that it’s hard to have focus when there’s so much you want to do, but lack of focus is the bane of progress.

But I do want to be more disciplined, which includes setting and following deadlines, something I’ve never been very good at, especially with more creative-oriented projects like writing.  But… I have to.

I’m not sure how this is going to work out, but here’s my current plan: My book on my theory of melody will be my top priority.  I’m setting the deadline of having an eBook version of it available on Amazon Kindle by November 25th (my birthday).  Meanwhile, I’ll set smaller piecemeal deadlines and goals for the other projects each week.  I’ll see if that will help me accomplish anything, or if I’ll need even more focus than that to make progress.

I’ll probably also try to blog more, because I think blathering about updates on my progress helps me actually make some progress because it gives me something to blog about.  Weird circular psychology, but it works, like making a to-do list and crossing things off…

Oh, and I’ve still got some co-authoring projects going on, which are exciting, so they will be stealing away time as well.

Anyway, for the remainder of this week, my goals will be to:

  • Plan the “book on melody”, breaking down the project into smaller chunks that can be given deadlines of their own
  • Finish plotting that fantasy book I’m working on called “Stormground”
  • Co-authoring stuff

I’d also like to try to use some of the money I made from that freelance programming gig to buy a new set of headphones.  I haven’t had a good pair of headphones to compose music with for over a year now (maybe even two years… or three… it’s been a while).  I usually use wireless headphones, but they’re not very good for composing because they make automatic volume adjustments, keeping things from being too soft or too loud.  Which is great when you’re watching a movie or just listening to music, but it’s terrible when you’re actually trying to compose it.  Plus, wireless headphones always have that faint white noise in the background, which is very annoying when you’re trying to hyper-focus on sound.  And my other headphones have pretty much worn out, or were cheap and terrible to begin with.  So: must buy new headphones.  Especially since composing some new music will be necessary to help advertise the melody book, and I want to start building interest for it as soon as possible.

By S P Hannifin, ago

Projects for June 2014

I haven’t blogged in a while, so here’s a little update of what I’m working on.

Writing fiction

I’m still working on co-authoring two short stories. We’re almost done with one. It’s taken us a lot longer than expected, but I always find interesting the seemingly random issues we find we have different perspectives on and end up discussing for hours. Maybe that’s not very productive writing-wise, but I actually really enjoy it; even if I don’t necessarily change my mind all the time, it forces me to think about things differently.

I’m still querying agents for my novel, Son of a Dark Wizard. I have yet to start another solo writing project, though I have a few stories in various phases of plotting. What I’m hoping to do with my next novel, whenever I start writing it, is to put it online for free, a bit like I did with The Game of Gynwig years ago, and then self-publish it. I even spent a week creating a custom WordPress theme and website to host the project at Morrowgrand, which will hopefully host other future projects as well so they’ll all cross-advertise each other. (Speaking of WordPress themes, the theme for this blog really needs to be updated at some point, doesn’t it?) In addition to posting the blog online, I actually think it would be interesting to record (with screen-capture software) my entire writing process of the novel. I know I’m not the first to try that, but it seems like a fun idea. (Although I guess if I do that, I won’t be able to write long-hand, as I do roughly half the time.)

On top of writing and releasing the novel, I also hope to write a musical soundtrack for it.

That said, any solo-writing projects may take a back seat to my other projects:

Melody project stuff

I’m finally returning to my book on composing melodies that I’ve been trying to write on and off for years. I think I’ve finally found an interesting angle to take with the book. I was struggling because my attempts were coming off as really academic and bland. Finally I thought: why not treat it like Johann Fux’s Study of Counterpoint? His book features two characters, a teacher and a student, talking about the subject. So, rather than a straight-forward non-fiction here’s-a-bunch-of-information sort of book, I can wrap the instructional material in a sort of two-character story arc. At the very least, it would be much more interesting for me to write. So that’s one project.

The other project is the real Parnassus: the symphony generator. A piece of software that uses the algorithms of my melody generators to help automate the entire composition process. The idea is that the symphony generator would be able to compose entire symphonies on its own, automatically, with the composer providing as much or as little creative input as he wishes. My two main goals for it, that I’ve been daydreaming about for over a decade: 1) I want it to be able to compose a Mozartean symphony at the touch of a button (or maybe a few buttons, if I must compromise), and 2) I want to be able to give it only a melody, and have it flesh it out into a full piece with little or no input from me. Obviously I want it to be able to do more than that, but those are my main two objectives.

I’d like to get them both out by December, in time for the holiday season (and before I go completely bankrupt), but almost everything I do takes me longer than I expect (except for going bankrupt).

Updates

I never did write that screenplay last month. I just found I was more interested in whatever else I was working on at the time. Maybe next year.

And that Nickelodeon cartoon pitch a few months back never went anywhere, unfortunately. That is, Nickelodeon wasn’t interested. I still want to do something with my pitch material at some point (though probably no time soon), but I’m not sure what.

By S P Hannifin, ago