Programming

Note-taking app update

Fortunately the nightmares have stopped and I’ve managed to get on a more regular sleeping schedule, going to bed at about 1 AM or so and waking up at about 9:30 AM. So that’s good.

Programming-wise, I spent much of last week just learning more about modern web tools; I’ve been away from web development for a while, so there’s a lot to learn. I switched from Feathers and React to what’s called a “MEVN” stack, which consists of MongoDB for the database (which I find easier than MySQL1), Express.js for the back-end framework, Vue for the front-end (which I find easier to learn than React), and of course Node.js for the back-end platform. In particular, this little tutorial on setting up a MEVN stack was super helpful.

Beyond that I’ve only just started programming some actual functionality. At the moment, you can create a note and delete it. And that’s it. Pretty boring. Much left to do.

As seen above, when you hover over a note, you get that little side-menu with edit, copy, and delete options. Looks a bit bland, but the design is definitely not final. Here’s my current todo list before making the app public:

  • Complete functionality of the sub-menu (the delete works, but the edit and copy buttons don’t yet)
  • Allow attaching or inserting media (at least images) to notes
  • Sort / search notes by tags
  • Create note with tags pre-filled (like adding a post to a category)
  • Allow some js widgets in notes, at least checkboxes and timers
  • Allow creation of new notes with bracket links (e.g. “[[]]”) as in MediaWiki (standard wiki feature)
  • Open related notes by clicking on these bracket links (without opening a new page)
  • Search and sort notes by content / data / date (only basic queries for now)
  • Create user login / registration system
  • Create user stats page (notes created, storage space available, etc.)
  • Collect user usage stats (for studying how users use the product)
  • Overhaul / finalize designs, make responsive and usable on mobile
  • Write Terms of Service, privacy, cookie use policies
  • Offer subscriptions, launch!

I have some other big features I’d like to add, but I’d like to launch it first, just to get it out there. So how long will all that stuff take to do? Assuming I can tick one off every day, only two weeks. But some will be easy and some will be difficult, and you never know what task you might get stuck on. So we’ll see. I’m going to aim for two weeks. Though I do have some family stuff going on that will likely also get in the way.

Plus this really awesome guy sent me the PS4 game Dreams that I’m eager to try…

By S P Hannifin, ago
Programming

Startup programming update

Here’s a quick update of what I’m working on…

First I’ll admit that I’m in a bit of a mess sleep-schedule wise; I don’t know what’s going on with my brain. I’ll sleep for 10 hours then stay awake for 10 hours or less, getting suddenly really tired at seemingly random times, constantly out-of-whack with a 24-hour schedule. And I keep having weird nightmares so my subconscious must be a bit anxious about something. (Part of what makes them weird is that they’re often emotional nightmares; I’ll be really angry or sad or depressed in the dream, rather than just having bad stuff happen to me in the dream. But I can’t find much information about emotional distress within dreams; most articles talk about emotional distress in life causing nightmares, but I’ve got the opposite problem. It’s bizarre.)

Anyway, I’m taking a short break from working on TuneSage to work on a quick related project. I’m not sure what I’ll call it yet, but it’s a note-taking / note-organizing web app. I mostly just want it for myself. I know there are already dozens out there (like Evernote), but none of them work quite the way I want them to. Specifically I want an app in which I can:

  • Keep and organize random notes / ideas
  • Keep todo lists and reminders
  • Keep and organize bookmarks (similar to what I use Pinterest for)
  • Keep and organize timed sessions (such as for freelancing)
  • Keep logs of stuff (such as books owned & read, movies owned & watched, TV shows watched, etc.)

All these things can already be done with existing tools, but again, not in quite the way I want.

Anyway, the tools I’m using to create this app are tools I want to learn for use with TuneSage, so it will ultimately help with TuneSage’s development too. Plus the app itself will help me organize my notes regarding TuneSage. So I’m trying to kill a bunch of birds with one stone.

I’ll of course release it and if it gains any traction, so much the better. If no traction, it’s still something I want for myself.

As far as TuneSage goes, I’ve stopped using Rust for now, and instead switched to the older but more popular Node.js. Node.js is mainly Javascript for back-end, and it’s much easier for me to program in an easy language I already have experience with than learn a new more complicated language like Rust. (For instance, I love not having to worry about variable types… most of the time.) And since Node.js has been around for a lot longer, there are a ton of resources out there for it, which is also a huge plus.

For this new mini-sidequest of a note-keeping app, I’m learning some other tools that I think I’ll need for TuneSage anyway, so the note-keeping app will be a good learning experience. I’m currently learning React for the front-end, which helps with creating user interfaces, and Feathers for front-end to back-end communication, which is what I’m finding more confusing at the moment, mostly because it seems to have a lot of “parts”.

So that’s what I’m up to at the moment! Stay tuned for the release of the first version of the note-keeping app!

By S P Hannifin, ago
Programming

Music Editor Developer’s Log: More Cowbell

I got the soundfont to work, or at least to work well enough for my prototype-creating purposes. It will need some fine-tuning in the future, but if I can manage to actually turn this software into a business, it would be nice to create a custom soundfont for it anyway.

I’m now almost to the point where I can start using this software to actually write some music, but I’ve still got a number of controls and GUI elements (buttons and stuff) to program. I need to add the abilities to do mainly the following:

  • add and delete measures
  • edit note / track variables such as
    • release time (how long it takes an instrument to fade away after it has stopped playing)
    • volume / velocity
    • stereo position (left or right)
  • edit reverb settings
  • save and load files
  • export and load MIDI files (depending on time; this feature isn’t too important yet)
  • export MP3 or WAV files (at least look into it; if this is too time-consuming, it’ll be something to look into in the future)

I think that’s mostly it. And none of that stuff (save for perhaps the last one) should be too terribly difficult to code. So I think I can get it done this week!

After that, I will probably be a bit more secretive as I begin adding the “secret ingredients” which are my amazing world-class AI music generating algorithms, which will be the secret sauce of the business. For that, I will probably have to buy a dedicated server (or VPS), as those algorithms will be executed server-side. That’ll be fun.

Hopefully I’ll also be able to use this editor to actually compose some new tracks this April. I owe my few Patreon subscribers probably around a dozen or so tracks, and I want to get that new album out, which just needs one or two more tracks. And it would just be a good test of the software, even without the AI features, to see what composing with it is like. 122 days left!

Oh, what exactly will constitute success come July 31st? I mentioned earlier that success will mean that the software will either be at a point in which it’s ready (or close-to-ready) to actually market and sell, or in which a working prototype is ready to show to investors. Of course, those possibilities are not mutually exclusive, but at least one must be the case. But what does the latter mean? What will make it “ready” to show?

Anything really, so I can’t lose!

Seriously, though, it will mean that the software should be able to auto-write a complete song (minus lyrics) on its own. That’s melody, chords, orchestration. The algorithms are done, it’s just a matter of making them usable to an end-user and making their output as good as possible.

I’d ideally like the software to be able to compose something with the complexity of a Mozart symphony. That would be the true peak of Parnassus. And I’m positive we’ll have that soon enough. Maybe not by July 31st, but it would certainly be awesome, no?

By S P Hannifin, ago
Programming

Music Editor Developer’s Log: Soundfont Insanity

For the past week, I’ve been trying to give my music editor1 the power of sound. I looked into the new Web MIDI API standards, but those are more for sending and receiving MIDI messages, not playing sound, so that’s no help. (Though it may be something to look into later for other features, of course.)

So instead I’ve been looking into the Web Audio API, which does the trick, and has mostly what I need. Actually, it has everything I need, but not everything I want. I want the sounds to sound as good as possible, which means the instrument samples must loop for sustains (as a MIDI synth would).

First I experimented with MIDI.js‘s implementation of sample playing. With pre-rendered soundfonts, I could easily play samples for all the basic MIDI instruments. Problem with this implementation is that the instruments don’t loop! (Or at the very least, they don’t seem to read in the looping data saved in the soundfont.) Instruments such as strings, which can sustain indefinitely, really deserve some decent looping.2

So I moved on to experimenting with a library called sf2synth.js. I can’t understand the Japanese comments (the developer seems to be from Tokyo), but this implementation seems to load in soundfont files much more completely, and actually reads in and uses the looping data! Woohoo!

But even it has a problem. When I play a note from the Musyng Kite soundfont  (which is the soundfont I’m currently using for experimental purposes) in the Polyphone Soundfont Editor (which is a great piece of software), it sounds great. But when it’s played back in the browser through sf2synth.js, it sounds more bland.

Here is what I think is happening…

If we look at a preset in Polyphone, we can see that it’s actually made up of multiple instruments; below you can see that “Strings Ensemble” is actually made up of 8 layers.

To me, it sounds like sf2synth.js is only playing one of these layers, instead of all of them like a true soundfont player should.

So my mission for this week is to dig into the sf2synth.js code, try to understand how it’s loading and playing sounds from the soundfont file, and try to give it the ability to play all the layers in a preset that it should. Polyphone is open-source, so I can also dig around their code to see how they’re loading in and parsing / interpreting sf2 files.

I probably only want to spend two weeks max on this; if I can’t figure it out after two, I’ll just have to settle for suboptimal sounds and move on. I can always come back to soundfont programming later. It’s more important to get a working prototype finished by the end of July. 129 days left!

By S P Hannifin, ago
Programming

Work on web MIDI editor continues…

Progress on my web-based MIDI editor / animator has been slower than I’d like, but isn’t that always the case? At the moment, I’ve got the basics I want; you can add and delete notes, copy and paste, create and delete tracks, hide and show tracks, and edit track colors. Still need to allow you to add and delete measures though.

But what I want to work on next is the sound; my editor doesn’t actually play any sound yet. I may perhaps try and utilize this javascript soundfont player: https://logue.dev/smfplayer.js/ … of course, soundfonts don’t sound nearly as great as sample libraries, but until someone programs a javascript-based VST host that a browser can use, I’ll have to settle for what I can find. Users should be able to import and export MIDI files anyway.

136 days left until July 31st!

By S P Hannifin, ago
Programming

GUI programming drags on for third week

Checking off some items from the to-do list mentioned in my previous post, my OpenGL GUI panels can now have rounded corners (although they still need a bit of work as they’re not quite as circular as I’d like), they can have shadows of arbitrary size and color, they can have linear and radial gradient coloring, and the text dynamically word-wraps as necessary, with the scroll bar appearing only when needed.

Ain’t that nice? Next I need to work on vertical resizing and making sure multiple panels will overlap correctly when on the screen; a panel should be brought to the front “layer” when clicked on.

Then I will move on to adding the GUI elements I mentioned in the last post.

A fellow programmer on twitch mentioned a GUI library for LWJGL the he had programmed called legui. Looks quite nice so I might play around with it and see if I can integrate any of it with my panels. Another advantage of streaming on twitch: meeting other programmers and discovering new things!

(P.S. I don’t really know if this is my third week of working on it or not, but it’s probably around there. The title of this post is a reference to a film in which people carry in the banner.)

By S P Hannifin, ago
Programming

My GUI programming efforts continue

I’ve been continuing my efforts to program a custom simple 2D/3D engine in Java with OpenGL with which to create a more user-friendly standalone version of my MIDI animator (and hopefully some games in the future). Lately I’ve been focusing on creating a GUI system. In terms of free open-source 3D engines in Java, I don’t much like most of the GUI implementations, so creating my own has been fun. Granted, I still run in to bugs that are tricky to fix while trying to accomplish relatively simple things, but trying to solve such problems is educational and always a bit addicting, like solving a puzzle, especially as I’m doing it for myself.

So here’s my latest progress update. I’ve settled on using STBTT (included in lwjgl) font rendering for the GUI text and the underutilized Nvidia path rendering (included in OpenGL in general) for the panels (and hopefully more in the future, including possible note shapes in the actual MIDI animator). The panels are movable and can contain an arbitrary amount of word-wrapped text. You can scroll through the text, and the panel can include a y-axis scrollbar of arbitrary size. You’ll also notice that the text fades in and out at the top and bottom, a nice little touch in my opinion that I probably spent too long programming:

I’ve also been streaming my programming efforts on twitch, which is also addicting. It turns what is usually a pretty solitary activity into something at least a little more social, and it’s fun watching the viewer count tick up now and then. Shout out to twitch user Subtixx who has even offered helpful points and advice allowing me to fix problems my eyes don’t catch (such as null-checking the darn scrollbar!); many thanks Subtixx!

My current to-do list includes (but is not limited to):

  • allow gradient coloring on the panels and text
  • allow rounded corners on the panels
  • allow panels to have shadows (render a semi-transparent panel behind)
  • allow panels to be resized dynamically
  • allow panels to be closed (and maybe shrunk?)
  • allow panels to contain some select common GUI elements, such as:
    • buttons
    • text fields (short and long)
    • radio buttons
    • check boxes
    • sliders (continuous and quantified)
    • menus (pop-up and drop-down)
    • tabs on the top or side of panels (which perhaps are just specialized buttons?)

So there’s still quite a bit to do. But I must do it, for this is my quest.


In other unrelated news, a 4th album from symphonic / power metal band Ancient Bards is finally happening! I love this band and I can’t wait for their new album!

By S P Hannifin, ago
Programming

Font rendering: stb oversampling vs NV path rendering

I told you I wanted to try stb oversampling in my last post, and I did! Here is the result (stb 2×2 oversampling on top, nanovg in middle (which is based on stb anyway, but no oversampling), and NV path rendering on bottom, screen size then 5x zoom):

So stb oversampling definitely looks the best, although it is still pretty fuzzy. And moving it around by subpixels looks very decent; it doesn’t get very much of the “shimmer” effect. (Still a bit, but not much to be bothersome.)

Here’s stb oversampling (top) vs NV path rendering (bottom) with a bigger font size (screen size then 2x zoom):

Here, I think NV path rendering looks better; it’s definitely less fuzzy. (The trade-off is that it does suffer from more “shimmer” when translated by subpixels, but it doesn’t bother me too much.)

You can also see that NV path rendering is able to utilize proper kerning: the ‘e’ is slightly below the capital ‘T’, as it should be. Each letter isn’t being drawn on a textured quad, so overlapping is trivial. (Well, for the end library user, at least.)

So, I think that completes my foray into font rendering for now. I’m too lazy to make bitmap fonts at the moment; stb oversampling will work for smaller fonts for now. Time to continue on to other GUI elements. I will try to design the GUI system in such a way that it will be able to utilize any font rendering system should I wish to create bitmap font support in the future.

By S P Hannifin, ago
Movies

More font fun and other random stuff

Font rendering in OpenGL

Haven’t done so much programming in the past week, but I did try rendering fonts with NanoVG (in lwjgl). Unfortunately it’s really not much better than just using OpenGL’s NV path rendering extension. Small fonts look slightly better, but not really good enough for me to want to use them. See the example below, a zoom-in of an 8-pixel high rendering of the font “Verdana”, NanoVG rendering it on top, NV path rendering below. NanoVG is better, but it’s still way too fuzzy to look any good.

So I might just use bitmap fonts for small text; I can’t see any alternative. (Bitmap fonts basically means loading in each letter as a pre-rendered picture and plopping it on the screen. The disadvantage is that they don’t look very good when resized or positioned between pixels, but they’ll at least be guaranteed to render small fonts clearly and crisply.) I’ll continue to use NV path rendering for larger fonts or fonts that need to be animated more dynamically or rendered with 3D perspective. I do want to try using oversampling with stb for small fonts before I move on from font rendering and further develop a GUI system; the demo doesn’t look too bad.


Kanopy film streaming service

I just realized our local library offers free access to the film streaming service Kanopy, and they’ve actually got a decent selection. Not the latest blockbusters, but some good foreign and classic cinema. (They’ve even got The Red Pill, the controversial anti-radical-feminist documentary which Netflix refuses to stream. (Though they did just recently finally add the DVD to their catalog.)) Since it’s free (for library card holders of participating library systems), we’re limited to 10 streams per month, but the streaming quality is decent. It’s not full HD (at least not on my PC), but it’s better than DVD quality. Interestingly they also allow you to embed videos. Here’s “Kumiko the Treasure Hunter”, which I really enjoyed (which I guess you won’t be able to see without an account, haha):

Interesting indeed!


Some metal music

Finally, I recently discovered the band Elvenking. I couldn’t quite get in to some of their older work, but their last album, released last year, is quite catchy. Disregarding the bizarre sense of fashion metal band members tend to share, this song is some power metal awesomeness:

I also came across the latest album from the symphonic metal band Leaves’ Eyes, and found it to be quite catchy as well. Love the use of choirs, the female lead’s operatic voice, and the cheesy fantasy lyrics. (I’m not a big fan of the growl singing, it sounds so gross and ugly and demonic, why is it so popular? It sounds so awful, so unmusical. Bah!)

OK, that’s all for now.

By S P Hannifin, ago
Programming

Font programming fun

So I’ve started working on GUI elements for my little engine that could. My quick list of things I’d like to include in a GUI system:

  • ability to create a menu bar
  • ability to create windows / panels with drop shadows
  • ability to create scroll bars for window content
  • ability to create buttons and button animations (hover, click)
  • ability to create text boxes and text fields

I started working on text today, i.e., font rendering. The NV path rendering extension actually does a pretty good job with larger text:

It’s not so good with smaller text:

Look at that ugly subpixel rendering:

Bleh! So I’ll have to figure out something else for smaller fonts. I’ll probably look into the Java AWT package, or perhaps look into NanoVG, especially since it looks like lwjgl already has bindings for it. (Maybe it already has everything I want?)

By S P Hannifin, ago