melody project

TuneSage has launched! (Update 15)

I finally launched the MVP version of TuneSage yesterday!

Less than two months ago I posted about finally honing in on a productive algorithm that made progress smoother. So less than two months from that to launching an MVP is pretty satisfying!

Granted, it’s just an MVP with the bare minimum of what I want it to be capable of, but one must start somewhere, and having a live version that people can try out as I continue to develop it should be useful in terms of getting feedback.

What’s not so satisfying is just how insanely crowded the AI music space has become in recent years, and most of it is, in my biased opinion, pretty awful. Suno is fun, but it focuses on full audio more than the individual notes. That surely has much wider appeal, but free open models are constantly on its heels; it will be interesting to see what becomes of it in 10 years; how much will it have to pivot to becoming an AI-powered DAW to survive?

The more MIDI-focused AIs out there are far less impressive, in my opinion, and that’s the niche I’d really like to fill. But battling them in the SEO and getting TuneSage noticed at all is probably going to be a long, boring, unforgiving trudge. Still, it must be done. Someone must do it. I shall journey forth.

So it’s launched, but that’s just the start. Up next on the roadmap is:

  • Continuing to fine-tune the generator so that it can begin to generate differentiated styles (I wanted to do this before launching, but realized that it would take too long)
  • Adding more accompaniment (more tracks supplying more harmonic and rhythmic material)
  • Creating full song templates so the generator can generate entire multi-track pieces

The grand vision is for TuneSage to be able to generate a full (and completely customizable) Mozartean symphony with a single click. Still a ways away, but should be perfectly possible eventually!

You can help me reach this dream… go subscribe to TuneSage today!

By S P Hannifin, ago

TuneSage progress update 14

TuneSage is almost ready to launch, at least as an initial MVP. On the todo list:

  • Fix a few bugs
  • Add a few user account features (account info type stuff)
  • Get the app working online (rather than just on my local computer)
  • Add some maintenance and analytics to the back end

Then it will finally be time for the business stuff:

  • Register as an LLC
  • Open business bank account and Stripe account
  • Integrate Stripe
  • Integrate some Terms of Service
  • Integrate a feedback / bug reporting field
  • And an FAQ and a news or blog section to the site

… and I think that’s it. Some of those business items may need approval time, so I probably won’t launch this week as I’d hoped, but fingers crossed for next week!

Oh, I also vibe-coded a little MIDI animator for sharing some TuneSage melodies (though the chordal accompaniment is still very generic for now):

By S P Hannifin, ago

TuneSage progress update 13

In my latest TuneSage devlog, I demonstrate the generator’s ability to compose different melodic forms, custom chord progressions, and partial rerolls for selected notes! Which, as mentioned in the last post, were pretty much the final features I wanted to include before launching…

I’m hoping to launch the web app in just a week or two, between July 31st and August 7th. I want to spend a few more days honing the default generic style it will launch with, then do an overhaul of the interface, polishing up its look and feel. I’ve also got to set up user accounts, logins, that sort of thing, but that shouldn’t be difficult.

It will be a SaaS, a subscription web app, monthly or yearly. At launch, I’ll probably offer an “early adopter” price for the first 100 or 250 users, perhaps $5 a month? And then upping to $9 a month afterwards. And then the fun of seeing how long it will take to get traction! (As I continue building the tool, of course.)

By S P Hannifin, ago

TuneSage progress update 12

I’ve recently been working on a “pop” style for melodies, but they still sound a bit generic at this point. (There are plenty of samples in the video below.) With melodic styles having a lot of overlap, I’m not quite sure how or where I’ll end up drawing distinctions. But I’m less focused on delineating styles at the moment, and more focused on broad capabilities, making sure new things it learns remain compatible with what it’s already learned. For now, it’s got two new “forms”: a 4-bar phrase that repeats, and a 2-bar phrase that repeats twice followed by a 2-bar closing. It’s also does some minor syncopation now and then, and I’ve added a setting that allows users to set the “note density” to fine, medium, or coarse. (Fine means: more syncopation, more 16th notes, sounds better at slower tempos. Coarse means the opposite: less syncopation, no 16th notes, sounds better at faster tempos.)

I also added “theme blocks”, the idea being that one can write a mockup for a whole song by stringing together multiple themes.

Still much work to do, including what I mentioned last time: allowing the user to “lock in” part of a melody and have TuneSage regenerate the rest. I need more forms, especially ones longer and shorter than 8 bars. I’d like to allow the user to generate chord progressions separate from the melody, or supply their own chord progressions.

Even with such a lot of remaining work to do, I’d really like to release the app as soon as possible, despite its initial shortcomings, just to see if anyone out there is even interested in paying for such a thing, and if so, get some real user feedback. I’m aiming for releasing it at the end of July or early August. I’m tempted to continually put it off later and later as I add more capabilities, but I could probably continually do that for years, and I’ve already spent way too many years just getting my initial music theory ideas to this current state. Stay tuned!

(P.S. I’m also making the devlog videos on YouTube to help me practice presenting, as I don’t think I am very good at it.)

By S P Hannifin, ago

TuneSage progress update 11

Haven’t blogged in a while! Does anyone read blogs anymore anyway?

Also haven’t made an update about TuneSage in a few years. But I’ve been working on it on and off, trying a lot of approaches and doing a lot of experiments that ultimately failed. Lots of trial and even more error.

But I finally have a completely overhauled algorithm that is both extremely fast and much easier to iterate on (unlike previous versions), so I’m excited and finally feel like I’m making good progress once more (despite the emergence of a vocal anti-AI crowd that didn’t much exist 10+ years ago).

Anyway, I’ve just made a YouTube video demonstrating TuneSage as it is at the moment:

As I mention in the video, the three main things I want to add before launching the product:

  • Melodic styles (right now the melodies are purposefully very random and generic sounding)
  • The ability to write melodies more or less than 8 bars long, and the ability to string melodies together for larger forms
  • The ability to edit individual notes or chords, and the ability to “lock” notes or chords in place while re-generating the rest

Plus, I’ll probably move the chordal accompaniment to a separate track so that the user can choose different instruments for both.

Still, the app will focus only on generating melodies with chords at first. I’ll work on expanding the accompaniment generation abilities after the app is actually launched.

(P.S. As of this writing, tunesage.com is down because I switched hosts and still haven’t reuploaded the landing page… I should do that soon.)

By S P Hannifin, ago

TuneSage progress update 5

It’s been over a year since my last TuneSage update, but work has been progressing. Work on the backend was slow and challenging; I spent some time going down quite a few dead-ends. But it’s actually generating melodies now, so I should be able to release some output samples sometime soon, fingers crossed.

I signed up for this year’s Y Combinator’s Startup School once again. I did it once back in 2019, but other than making a landing page (TuneSage.com), my progress was sparse. I just needed a lot more time than I imagined to make progress with the backend.

Startup School’s Course Guide says: “If you haven’t launched yet, make it a goal to launch during the program and get your first users!”

OK, I guess I’ll make it a goal then! Granted, I have repeatedly failed at making enough progress to launch whenever I have set it as a goal, but one must keep trying I suppose.

My question is: how many features do I need to launch with? For better or worse, my current plan is to just launch the product as a basic melody generator to start with.

So what do I need to do to launch?

  • Prepare the backend
    • Train the AI on more melodies (using public domain melodies)
    • Generate melodies in a variety of styles (these will be basic to start with)
  • Add at least some simple chordal accompinement features to frontend
    • e.g. root notes, arpeggio patterns, alberti bass, etc.
  • Overhaul frontend design (lots of tedious web design) and finalize
  • Figure out what soundfonts TuneSage will use
  • Figure out deployment and version control (honestly this can probably wait until after launch, but not too long after)
  • Create user account system
    • Create new account
    • Confirm email (if necessary)
    • Log in / out / reset password
    • Edit optional personal info
    • Usage stats
    • Terms and conditions
  • Register company
  • Find some payment system to use
  • Allow for a trial period (and decide exactly what that consists of)
  • Install some analytics system (so I can keep track of user engagement or whatever)
  • Stress testing? (In my experiments with “trovedex”, the database kept going down; I really don’t want that to be a problem. Anyway, I can always do invite-only if the system is too stressed, but that would be a good problem to have)
  • Launch! Update front page with information

Is that it? Am I forgetting anything?

Of course, there are many more features I’d still like to add; AI can be used for a lot more than just generating melodies. But it’s a starting point, and melodies are the one area I think other AI music systems struggle with the most at the moment.

So how long will all that take? Can I finish it in a few weeks? Startup School lasts for 7 weeks, so if I can do it in 3 or 4 weeks, that would be awesome. Considering how long things have taken me in the past, however, it will probably take me… 12 years. But for now let’s daydream:

  • Week 1: Finish backend and overhaul frontend
  • Week 2: Soundfont and user account system, start releasing samples
  • Week 3: Register company, install payment and analyctics systems
  • Week 4: Set up trial, stress testing, front page update, and launch!

That’s probably wishful thinking, but it’s better than nothing.

By S P Hannifin, ago

Music Editor Developer’s Log: More Cowbell

I got the soundfont to work, or at least to work well enough for my prototype-creating purposes. It will need some fine-tuning in the future, but if I can manage to actually turn this software into a business, it would be nice to create a custom soundfont for it anyway.

I’m now almost to the point where I can start using this software to actually write some music, but I’ve still got a number of controls and GUI elements (buttons and stuff) to program. I need to add the abilities to do mainly the following:

  • add and delete measures
  • edit note / track variables such as
    • release time (how long it takes an instrument to fade away after it has stopped playing)
    • volume / velocity
    • stereo position (left or right)
  • edit reverb settings
  • save and load files
  • export and load MIDI files (depending on time; this feature isn’t too important yet)
  • export MP3 or WAV files (at least look into it; if this is too time-consuming, it’ll be something to look into in the future)

I think that’s mostly it. And none of that stuff (save for perhaps the last one) should be too terribly difficult to code. So I think I can get it done this week!

After that, I will probably be a bit more secretive as I begin adding the “secret ingredients” which are my amazing world-class AI music generating algorithms, which will be the secret sauce of the business. For that, I will probably have to buy a dedicated server (or VPS), as those algorithms will be executed server-side. That’ll be fun.

Hopefully I’ll also be able to use this editor to actually compose some new tracks this April. I owe my few Patreon subscribers probably around a dozen or so tracks, and I want to get that new album out, which just needs one or two more tracks. And it would just be a good test of the software, even without the AI features, to see what composing with it is like. 122 days left!

Oh, what exactly will constitute success come July 31st? I mentioned earlier that success will mean that the software will either be at a point in which it’s ready (or close-to-ready) to actually market and sell, or in which a working prototype is ready to show to investors. Of course, those possibilities are not mutually exclusive, but at least one must be the case. But what does the latter mean? What will make it “ready” to show?

Anything really, so I can’t lose!

Seriously, though, it will mean that the software should be able to auto-write a complete song (minus lyrics) on its own. That’s melody, chords, orchestration. The algorithms are done, it’s just a matter of making them usable to an end-user and making their output as good as possible.

I’d ideally like the software to be able to compose something with the complexity of a Mozart symphony. That would be the true peak of Parnassus. And I’m positive we’ll have that soon enough. Maybe not by July 31st, but it would certainly be awesome, no?

By S P Hannifin, ago

Music Editor Developer’s Log: Soundfont Insanity

For the past week, I’ve been trying to give my music editor1 the power of sound. I looked into the new Web MIDI API standards, but those are more for sending and receiving MIDI messages, not playing sound, so that’s no help. (Though it may be something to look into later for other features, of course.)

So instead I’ve been looking into the Web Audio API, which does the trick, and has mostly what I need. Actually, it has everything I need, but not everything I want. I want the sounds to sound as good as possible, which means the instrument samples must loop for sustains (as a MIDI synth would).

First I experimented with MIDI.js‘s implementation of sample playing. With pre-rendered soundfonts, I could easily play samples for all the basic MIDI instruments. Problem with this implementation is that the instruments don’t loop! (Or at the very least, they don’t seem to read in the looping data saved in the soundfont.) Instruments such as strings, which can sustain indefinitely, really deserve some decent looping.2

So I moved on to experimenting with a library called sf2synth.js. I can’t understand the Japanese comments (the developer seems to be from Tokyo), but this implementation seems to load in soundfont files much more completely, and actually reads in and uses the looping data! Woohoo!

But even it has a problem. When I play a note from the Musyng Kite soundfont  (which is the soundfont I’m currently using for experimental purposes) in the Polyphone Soundfont Editor (which is a great piece of software), it sounds great. But when it’s played back in the browser through sf2synth.js, it sounds more bland.

Here is what I think is happening…

If we look at a preset in Polyphone, we can see that it’s actually made up of multiple instruments; below you can see that “Strings Ensemble” is actually made up of 8 layers.

To me, it sounds like sf2synth.js is only playing one of these layers, instead of all of them like a true soundfont player should.

So my mission for this week is to dig into the sf2synth.js code, try to understand how it’s loading and playing sounds from the soundfont file, and try to give it the ability to play all the layers in a preset that it should. Polyphone is open-source, so I can also dig around their code to see how they’re loading in and parsing / interpreting sf2 files.

I probably only want to spend two weeks max on this; if I can’t figure it out after two, I’ll just have to settle for suboptimal sounds and move on. I can always come back to soundfont programming later. It’s more important to get a working prototype finished by the end of July. 129 days left!

By S P Hannifin, ago

Y Combinator

Y Combinator looks like something I’d really like to do. Looks like it would be perfect for the symphony generator app I’ve been working on. As they say on their site:

Twice a year we invest a small amount of money ($120k) in a large number of startups.

The startups move to Silicon Valley for 3 months, during which we work intensively with them to get the company into the best possible shape and refine their pitch to investors. Each cycle culminates in Demo Day, when the startups present their companies to a carefully selected, invite-only audience.

The deadline for their upcoming Winter 2019 cycle is October 2nd, less than two months away. Unfortunately I’m not so sure I can finish a working prototype by then. Worth a try though. Might have to wait until their next cycle after that. Regardless, this looks like something to aim for!

By S P Hannifin, ago

More new music and what happened to the melody generator

I uploaded a new piece of music to YouTube earlier today called The Stormbringer’s Apprentice:

I write in the description:

This piece provides the themes for a villain called Stormov, from my book Insane Fantasy. He’s an apprentice for the mysterious “Stormbringer”, and helps to entice new recruits for his master’s evil plans. His themes are mostly dark, but there is a hint of adventure in there as well, as he lures potential helpers with temptations of worldly power.

Also, this is my first video to feature my own attempt at programming my own custom “music animation machine” programmed in Java with jMonkeyEngine. The frame rate is not as smooth as I’d like it to be (I’m still screen-capturing it), but it’s not horrible at least. Anyway, I think I’ll enjoy playing around with it. Obviously it’s inspired by Stephen Malinowski’s work, as I’ve been using his decade-old “Music Animation Machine” program for a long time now.

As mentioned above, this piece is animated with my own custom “music animation machine.” I may release the source code for it at some point for anyone who’s interested in it, but you’d have to download jMonkeyEngine to compile it and run it yourself; I’m not really interested in making into a standalone program right now, as I hope to continue fooling around with the code. The code is also sloppy and contains features I never finished programming, as it’s all part of an ongoing MIDI sequencer project. Also, it doesn’t actually even play MIDI files… it only makes the shapes and animations based on a MIDI file, which is all I need it to do as I sync videos with tracks recorded from Overture in Windows Movie Maker before adding titles and uploading to YouTube.

In case you missed it, last week I posted this track called Storybook Overture:

So that makes five tracks completed so for my upcoming album (which I have no idea what I’ll name yet): Lullaby of the Westwind Woods, The Storm Cometh, A Stargazer’s Lullaby, Storybook Overture, and The Stormbringer’s Apprentice. Together they amount to over 20 minutes, so I’m over 1/3rd finished the album!

In other news, I’ve had several people email me over the last few months asking about whatever happened to the melody generator. Well, I was contacted by a shady underground group that secretly controls the world, and they warned me that the world simply is not ready for something so powerful.

Actually, the project just got to be too frustrating. It was getting enough interest that people were thirsty to learn how it worked, yet not enough interest to fund a Kickstarter, and I just didn’t (and don’t) have the time to give it the attention it needs. I of course haven’t given up on it, but it’s on the back-burner for now until I can get my life sorted a bit more. In the meantime, I’d just rather spend my free time writing music and books and programming games, as those projects tend to actually generate some income. Not much, but something. So I have absolutely no idea when I’ll be able to get back to working on the melody generator. Could be later this year, could be two or three years down the road.

I have an email list here if you’d like to be updated when I actually return to the project: Melody Generator News.

By S P Hannifin, ago