Tunesage

TuneSage progress update 10

To my eternal shame, it’s been some months since I made any decent progress on TuneSage. But I’ve been back at it in the last few weeks, trying to tackle the time-consuming problems I’ve been having. Clearly my initial plans were not practical. Here are my current plans:

The AI

I’m vastly simplifying the “AI” element. In fact, I might even stop using “AI” to describe the app altogether. It’s become an overused marketing buzzword in the last couple years anyway. Users will still be able to generate melodies automatically, of course. But the backend will be a lot less complicated.

So I’m rethinking the whole concept of musical styles. My initial plan was simple enough: feed musical examples into a neural network, have it identify styles, and then use it to help write new music in those styles, pairing it with the melody-generating algorithm I already have. But that’s just not working very well, and I’ve spent way too much time fooling around with that approach.

But what exactly is musical style anyway? For melodies at least, we can probably get similar results by simply identifying and using melodic tropes, or signatures, and avoiding melodic rarities for a particular style. And on the melodic level, such tropes are simple enough that they can be identified and implemented without needing to train anything. Instead, we can just say, “hey, melody generator, make this melodic trope more likely to occur in what you generate.” Done. Easy.

Anyway, for the sake of just getting this darn app launched and getting a minimum viable product out there, I think I’m going to ignore styles for now altogether.

The front-end

I’ve been having difficulty figuring out just what the front-end should look like and how it should work.

Firstly, the app will focus, at least for now, only on writing or generating melodies. It won’t be for composing and mixing entire pieces, not at first anyway, unless they’re extremely simple. So, because the paradigm is focused on writing tunes, the traditional piano roll view or the track view, both of which I’ve spent some time putting together, just feel too clunky for editing melodies. The whole point of the TuneSage app is to change the paradigm of composing music, at least melody-wise, so it needs a view / layout designed for that purpose.

So I think I’ve finally come up with something that might work, which I’ll reveal when I get closer to launching (or on Twitch if / when I stream my programming again).

The current to-do list

  • Front-end
    • Buttons for: create new melody, generate melody, delete melody, move melody
    • Set tempo option
    • Allow user to “lock” notes & chords to allow for regenerating only a part or parts of a melody
    • Chordal accompaniment templates (mostly already done)
    • Chord chooser options (mostly already done)
    • Export MIDI / Save / Load options
    • Melody options
      • Time signature (probably only 2/3, 4/4, 3/4, 6/8 to start)
      • Key signature
      • Instruments for melody and chordal accompaniment
      • Volume
    • Play functionality (play, pause, stop)
    • Demo settings (not sure what the limits should be yet… perhaps limited time, no MIDI export, can only create a certain amount of melodies? Also need to find a way to discourage bots.)
  • Back-end
    • Melody generation code (mostly already done)
  • Overall app stuff
    • User login system
    • Terms of service page
    • Subscription service (Stripe?)
    • Create landing page
    • Actually incorporate as a company
    • LAUNCH

I think that’s it. Lots of stuff, but should all be doable, especially as I’m going to stop fooling around so much with the backend AI stuff for now.

By S P Hannifin, ago

TuneSage progress update 9

Getting closer to launching an initial version of TuneSage. I obviously bit off more than I could chew in creating the front-end. It still lacks a ton of features I’d like, but it’s just taking too long to program. I should’ve started with something much simpler. Oh well, too late now, I’m almost done. Here’s what I have left to do:

Front-end:

  • Delete track button
  • Copy / cut / paste sections
  • Melody / phrase options
  • Edit key signature
  • Ctrl+z undo
  • Export MIDI file
  • Finalize soundfont
  • Finalize icons

Back-end:

  • Use melody / phrase options sent from front-end
  • More training / styles

Overall site:

  • Create user / login
  • Boilerplate terms of service
  • Integrate user subscriptions (decide on a payment collector)
  • Create a landing page
  • Actually incorporate

I might just skip the front-end Ctrl+z stuff for now, because I’m not quite sure how to implement it efficiently and I’m afraid it might be too time consuming. Otherwise, I think I can finish the front-end stuff by the end of next week.

I’m not sure how long the back-end training will take, because it takes of a lot of tedious data formatting and trial and error.

And the final overall site stuff shouldn’t take long. I’ve never integrated a subscription service before, but I’m guessing most payment services make that pretty easy these days.

I’d love to launch before the end of the month. We’ll see how it goes!

By S P Hannifin, ago

TuneSage progress update 8

There’s still a lot of work to do, but I think I just might actually be able to launch TuneSage’s MVP this month! I’m done with all the difficult and time-consuming backend algorithmic programming. All that’s left with the backend now is manual training and calibration. That will still take some time, but it’s mostly data entry and testing. I think the melodic styles TuneSage will offer on launch will consist of:

  • Irish / Scottish folk song
  • American folk song
  • Sea Chantey
  • Hannifin (trained from my own compositions)
  • Generic

Of course, I’ll later add more styles, but these are easy to train for, as I can use simple public domain examples as training data, and they’ve already stood the test of time.

Still lots of stuff on the todo list though:

  • Melody generation options GUI on the frontend
  • Overhaul frontend design
  • Finalize soundfont
  • Allow setting track volume1
  • Recreate the landing page / home page
  • Add user account system (register / login / logout)
  • Add some basic terms of service
  • Setup payments system / trial period2
  • Setup some sort of analytics to track usage metrics
  • Register company (officially found it as a company)

So still quite a bit of work, but it’s all very doable, and I know (or am confident I can easily figure out) how to do it.

Then, after launching, I’ll be focused on iterating, iterating, iterating repeatedly, trying to improve the app and get users and feedback!

Stay tuned, I should be able to release some example melodies soon!

By S P Hannifin, ago

TuneSage progress update 7

The time is flying by too quickly! But I am making progress. The backend melody-generating code is working much better now, though it’s actually only writing four-bar phrases at the moment. So I’ll be working on expanding that capability for the rest of the week, as well as expanding its stylistic palette with more training data. (A melody is just a collection of related phrases, so the foundation is already there.) If I’m lucky, I may even be able to share some example output next week.

Frontend-wise, I think the only other feature I need to work on for now is the ability to add, move, and delete tempos, which should only take a couple of hours. The frontend it still needs a design overhaul, though, which will take another day or two.

The frontend will be missing a lot of features on launch, but users should at least be able to generate tunes and export them as MIDI files to open in their favorite DAW or notation program or whatever.

So I think my schedule is close to the same as it was in my last progress update:

  • This week: Finish backend and redesign frontend
  • Next week: Soundfont and user account system, start releasing samples
  • Week 3: Register company, install payment and analytics systems
  • Week 4: Set up trial, front page update, and launch!

If I can actually accomplish that, I could launch as soon as August 15!

But of course that’s probably not going to happen…

Still, we’re getting closer and closer!

By S P Hannifin, ago

TuneSage progress update 6

My goal last week was to “finish backend and overhaul frontend”, which definitely did not happen.

My work on the backend unfortunately came to another dead end. I was trying automate the training of the AI so that I could just give it melodies and it would train on them with little oversight. It worked, but too inefficiently; it must be continually tweaked to work well, so the whole endeavor ends up taking even more time. For now, it seems it will be more efficient time-wise to train it manually. In other words, I think it will be more time-efficient to use supervised learning rather than unsupervised learning. (Which is perhaps an obvious outcome, but it was worth a try.)

So I’m already weeks behind! Schedule now looks like this:

  • This week and next week: Finish backend and overhaul frontend
  • Week 3: Soundfont and user account system, start releasing samples
  • Week 4: Register company, install payment and analyctics systems
  • Week 5: Set up trial, stress testing, front page update, and launch!

Admittedly, it will still likely take longer than that…

Questions about startup idea:

The last startup school webinar was about evaluating your idea for a startup, and included a number of good questions to ask yourself about an idea to help with that. Granted, I’ve already chosen an idea (AI music SaaS), but I thought it still might be interesting to answer the questions:

  1. Does your team have founder / market fit to work on this idea? I don’t really have a team, but yes, as a programmer and a music composer, I think I have good founder / market fit. The product is something I want for myself and would use.
  2. How big is the market for this idea today? I don’t know. There are other AI music services out there, but I don’t know how good their profits are. The market for music software in general, however, is huge.
  3. How big could it be in a few years? Again, I don’t know, but I haven’t seen any indication that it’s growing rapidly at the moment.
  4. What is the problem you hope this product will solve? Have you seen this problem first hand? How are confident are you that it’s actually a problem? For your users, how acute and frequent is the problem? Composing music can be time consuming; there are lots of creative decisions to make. It can also be difficult to get going if you’re just getting started or haven’t done it in a while. Yes, I have experienced this first hand. I know others have this problem as well, as my previous melody generating apps attracted some users. As for how frequent and acute the problem is, I don’t know; I’ll have to talk to more users.
  5. Do you have entrenched competition? If so, how will you beat them? Yes, there are a few competitors in AI music. I need to do some more research on them, but in my opinion, they’re of limited use, particularly because they do not generate interesting melodies. They’re output tends to sound either too random or too bland. My focus on melody may be a good starting point to beat them.
  6. Is this something you personally want and would use? Definitely!
  7. Did this idea only recently become possible, or only recently become necessary? Yes and no. The algorithms would certainly be possible to run on older computers, but they take time to come up with.
    1. If not, why has no one solved it before? The algorithms are not obvious, I suppose, even with popular modern AI paradigms.
  8. What are the proxies – large, successful companies that do something similar to this? I don’t know if there are any. None that I know of, anyway.
  9. Is this a problem that you personally care about? Is it something that you would be willing to work on for a long time? Yes and yes.
  10. Can your solution scale? Could this be a consulting business in disguise? Since it’s an SaaS, yes, scaling is possible.
  11. Is this idea in a good “idea space”? I think? I’m not really sure what the “idea space” for AI music is.
  12. How did you come up with this idea? Did you start with the problem or the solution? Started with the problem. In fact, don’t even have solutions to all the problems yet! But I think the solutions are reachable.
  13. Do you have a new insight about this idea, one that few others have? Yes, I think my approach to generating melodies is a new insight; I don’t see any other service offering decent melody generation at the moment.
  14. What are the current alternatives that people use instead of your product? Why will people switch to your product? How difficult will it be to get them to switch? I admittedly don’t know; I’ll have to talk to users.
  15. How will you make money? SaaS!
  16. If this the kind of business that has a chicken-and-egg problem (i.e., a marketplace, a dating site), how will you solve it? No chicken-egg problem here!
    By S P Hannifin, ago

    TuneSage progress update 5

    It’s been over a year since my last TuneSage update, but work has been progressing. Work on the backend was slow and challenging; I spent some time going down quite a few dead-ends. But it’s actually generating melodies now, so I should be able to release some output samples sometime soon, fingers crossed.

    I signed up for this year’s Y Combinator’s Startup School once again. I did it once back in 2019, but other than making a landing page (TuneSage.com), my progress was sparse. I just needed a lot more time than I imagined to make progress with the backend.

    Startup School’s Course Guide says: “If you haven’t launched yet, make it a goal to launch during the program and get your first users!”

    OK, I guess I’ll make it a goal then! Granted, I have repeatedly failed at making enough progress to launch whenever I have set it as a goal, but one must keep trying I suppose.

    My question is: how many features do I need to launch with? For better or worse, my current plan is to just launch the product as a basic melody generator to start with.

    So what do I need to do to launch?

    • Prepare the backend
      • Train the AI on more melodies (using public domain melodies)
      • Generate melodies in a variety of styles (these will be basic to start with)
    • Add at least some simple chordal accompinement features to frontend
      • e.g. root notes, arpeggio patterns, alberti bass, etc.
    • Overhaul frontend design (lots of tedious web design) and finalize
    • Figure out what soundfonts TuneSage will use
    • Figure out deployment and version control (honestly this can probably wait until after launch, but not too long after)
    • Create user account system
      • Create new account
      • Confirm email (if necessary)
      • Log in / out / reset password
      • Edit optional personal info
      • Usage stats
      • Terms and conditions
    • Register company
    • Find some payment system to use
    • Allow for a trial period (and decide exactly what that consists of)
    • Install some analytics system (so I can keep track of user engagement or whatever)
    • Stress testing? (In my experiments with “trovedex”, the database kept going down; I really don’t want that to be a problem. Anyway, I can always do invite-only if the system is too stressed, but that would be a good problem to have)
    • Launch! Update front page with information

    Is that it? Am I forgetting anything?

    Of course, there are many more features I’d still like to add; AI can be used for a lot more than just generating melodies. But it’s a starting point, and melodies are the one area I think other AI music systems struggle with the most at the moment.

    So how long will all that take? Can I finish it in a few weeks? Startup School lasts for 7 weeks, so if I can do it in 3 or 4 weeks, that would be awesome. Considering how long things have taken me in the past, however, it will probably take me… 12 years. But for now let’s daydream:

    • Week 1: Finish backend and overhaul frontend
    • Week 2: Soundfont and user account system, start releasing samples
    • Week 3: Register company, install payment and analyctics systems
    • Week 4: Set up trial, stress testing, front page update, and launch!

    That’s probably wishful thinking, but it’s better than nothing.

    By S P Hannifin, ago

    TuneSage progress update 4

    It’s already been two weeks since the previous update? Bleh, I program too slowly. The undo / redo system was a bit tricky to implement, as was exporting MIDI files. The saving and loading pieces from a MongoDB database is also tricky, but that’s mostly because I’m having to reacquaint myself with how it works. It should only take about another day’s worth of work to finish that up, I hope.

    I’ve added just one more thing to the list that I want to finish before I start focusing more on the back-end, and that’s implementing chord markers. The first thing I’ll implement on the back-end is a chord progression generator, since that’s pretty easy (and hardly innovative), so of course I’ll need a way to display them.

    After chords, I can move on to melodies and accompaniment, which is where I hope things will get more interesting.

    • Playing features
      • Play from a selected position ✓
      • Play or loop a selection (skip!)
      • Show play location in minutes / seconds (skip!)
      • Pause play (saves play position) ✓
      • Show / hide measure numbers ✓
    • Move / create tempos (or tempo groups for accelerandos?) (skip!)
    • Note editing
      • Move / create / delete / edit notes or note groups (skip!)
      • Copy / paste notes or note groups (skip!)
      • Edit note velocities (skip!)
    • Measure editing
      • Delete / create / edit measures ✓
      • Delete / create / edit sections (measure groups)
    • Track editing
      • Load track instrument from MIDI file ✓
      • Edit track instrument ✓
      • Edit track volume / stereo positions (skip!)
      • Auto-color tracks differently (skip!)
    • Chords
      • Create / delete chord marks (only triads and 7ths for now)
      • Color lines by chord voice
    • Load / display / edit key signatures (skip!)
    • MIDI loading
      • Load note velocities from MIDI ✓
      • Fix MIDI loading bug: Extra note at end of file (skip!)
    • Saving / exporting
      • Save / load music pieces to / from personal database
      • Import / Export MIDI files ✓
      • Export music to WAV or MP3 (skip!)
    • Bookmark sections of a piece (skip!)
    • Zoom in and out (horizontally) ✓
    • Undo / redo support (Ctrl+Z) ✓
    • Import soundfonts from computer (low priority) (skip!)
    • Account
      • Confirm email (if necessary) (skip!)
      • Log in / out / reset password (skip!)
      • Edit optional personal info (skip!)
      • Usage stats (skip!)
    • Final design pass (make everything look good) (skip!)
    By S P Hannifin, ago

    TuneSage progress update 3

    A few more things checked off the list. Added a couple “measure editing” tasks. Quite a few tasks I think can probably be skipped for now, as I’d like to get back to working on the back-end so I can then work on integrating the back-end with the front-end. Most of the skipped items should not be overly difficult anyway (though difficulty is always easy to underestimate). Excluding the skipped items leaves the items in bold: measure editing, save / load / export stuff, and an “undo” system, which I guess will be better to tackle earlier than later.

    I still need to have a good think about what features the back-end should include at launch; that is, what specific music-generating features would be most useful. I’ll do that in another post a bit later, I suppose.

    • Playing features
      • Play from a selected position ✓
      • Play or loop a selection (skip!)
      • Show play location in minutes / seconds (skip!)
      • Pause play (saves play position) ✓
      • Show / hide measure numbers ✓
    • Move / create tempos (or tempo groups for accelerandos?) (skip!)
    • Note editing
      • Move / create / delete / edit notes or note groups (skip!)
      • Copy / paste notes or note groups (skip!)
      • Edit note velocities (skip!)
    • Measure editing
      • Delete / create / edit measures
      • Delete / create / edit sections (measure groups)
    • Track editing
      • Load track instrument from MIDI file ✓
      • Edit track instrument ✓
      • Edit track volume / stereo positions (skip!)
      • Auto-color tracks differently (skip!)
    • Load / display / edit key signatures (skip!)
    • MIDI loading
      • Load note velocities from MIDI ✓
      • Fix MIDI loading bug: Extra note at end of file (skip!)
    • Saving / exporting
      • Save / load music pieces to / from personal database
      • Export music to MIDI
      • Export music to WAV or MP3 (skip!)
    • Bookmark sections of a piece (skip!)
    • Zoom in and out (horizontally) ✓
    • Undo / redo support (Ctrl+Z)
    • Import soundfonts from computer (low priority) (skip!)
    • Account
      • Confirm email (if necessary) (skip!)
      • Log in / out / reset password (skip!)
      • Edit optional personal info (skip!)
      • Usage stats (skip!)
    • Final design pass (make everything look good) (skip!)
    By S P Hannifin, ago

    TuneSage progress update 2

    It’s been five days and, judging by the to-do list, it doesn’t look like I’ve done much. I actually spent a bunch of time fixing various bugs.

    Also I added a few new things to the to-do list; “auto-color tracks differently” will be easy; that’s just a matter of cycling through hues when a MIDI is loaded, the same sort of thing I did in the Java-based MIDI animator.

    I also want to allow the user to import their own soundfont from their computer. (Although honestly that’s probably a feature that can wait until after launch, so that one’s iffy.)

    Finally, we’ll need an “undo / redo” feature so that when the user presses Ctrl+Z, it will undo the last change. Not really sure of the best way to implement that, but we’ll figure it out.

    • Playing features
      • Play from a selected position ✓
      • Play or loop a selection
      • Show play location in minutes / seconds
      • Pause play (saves play position) ✓
      • Show / hide measure numbers ✓
    • Move / create tempos (or tempo groups for accelerandos?)
    • Note editing
      • Move / create / delete / edit notes or note groups
      • Copy / paste notes or note groups
      • Edit note velocities
    • Track editing
      • Load track instrument from MIDI file ✓
      • Edit track instrument
      • Edit track volume / stereo positions
      • Auto-color tracks differently
    • Load / display / edit key signatures
    • MIDI loading
      • Load note velocities from MIDI
      • Fix MIDI loading bug: Extra note at end of file
    • Saving / exporting
      • Save / load music pieces to / from personal database
      • Export music to MIDI
      • Export music to WAV or MP3
    • Bookmark sections of a piece
    • Zoom in and out (horizontally)
    • Undo / redo support (Ctrl+Z)
    • Import soundfonts from computer (low priority)
    • Account
      • Confirm email (if necessary)
      • Log in / out / reset password
      • Edit optional personal info
      • Usage stats
    • Final design pass (make everything look good)
    By S P Hannifin, ago

    TuneSage progress update 1

    There’s still lots of work to do on TuneSage. I’m hoping if I regularly blog some short progress updates, it may encourage productivity, as it did last year with Trovedex.1

    Right now I’m working on the front-end. As always, it’s a bit of a balancing act deciding what features need to be included and what features can wait to be added after launch. So far, the completed features include:

    • Load a MIDI from file
    • Mute / solo tracks
    • Change track colors
    • Show / hide tempo marks (from MIDI)
    • Edit / delete tempos
    • Show / hide measure lines
    • Show / hide pitch lines
    • Play MIDI

    Features on the to-do list:

    • Playing features
      • Play from a selected position
      • Play or loop a selection
      • Show play location in minutes / seconds
      • Pause play (saves play position)
    • Move / create tempos (or tempo groups for accelerandos?)
    • Note editing
      • Move / create / delete / edit notes or note groups
      • Copy / paste notes or note groups
      • Edit note velocities
    • Track editing
      • Load track instrument from MIDI file
      • Edit track instrument
      • Edit track volume / stereo positions
    • Load / display / edit key signatures
    • MIDI loading
      • Load note velocities from MIDI
      • Fix MIDI loading bug: Extra note at end of file
    • Saving / exporting
      • Save / load music pieces to / from personal database
      • Export music to MIDI
      • Export music to WAV or MP3
    • Bookmark sections of a piece
    • Zoom in and out (horizontally)
    • Account
      • Confirm email (if necessary)
      • Log in / out / reset password
      • Edit optional personal info
      • Usage stats
    • Final design pass (make everything look good)

    That’s not counting the further work of integrating the back-end with the front-end.

    Anyway, my goal is to finish all the above before the end of this month (April 2021), and then I can move on to integrating the back-end (which still needs a good bit of work itself).

    Feel free to suggest any features you’d like; though, like I said, I’m only trying to do the minimum needed to launch.

    By S P Hannifin, ago