Midi Game

This project was made to aid in learning to play piano. There are an abundance of midi files that can be found on the internet, spanning a very large number of songs. These files can be loaded in the program and the user can learn, play along, or just watch and listen. A midi keyboard can be plugged in and the program will read its inputs like a real piano.

In the settings, you can select which input and output device you would like to use. Input devices can include your computer keyboard, or a midi keyboard that you plug in. The output device is what synthesizes the sound that you hear. Most people will see the Microsoft GS Wavetable Synth. This is the default synthesizer that all windows computers come with. This synthesizer actually isn't very good and can cause a lot of latency, as it takes it time to create the sound. A midi keyboard can also function as an output if it can synthesize its own sounds. If you only have the default option, I recommend installing a separate synth, such as CoolSoft's Virtual Midi Synth. In my testing, I found that it can reduce the latency by about four times after reducing the buffer size. Not only is it a lot faster, but it sounds a lot better too since you can pick your own sound font.

Once you decide to play a song, you will be greeted by a file system. The file search system runs on a separate thread, so you don't have to worry about the front end freezing up. You can easily browse into and out of folders. Only midi and fff files will show up in this file system. fff files are an optimization that I will get into later. After selecting a song, it will be loaded into memory, and the song specific settings will be displayed.

Each midi file consists of a list of tracks. These tracks are a way that the composer has grouped notes together. They could be playing a different melody, they could be using a different instrument, or they could be grouped based off which hand you use to play it on the piano.

In the program, after a song is loaded, each track can individually be designated a hand to play it. It can be hidden to get rid of some clutter. Or it can be muted as well as hidden, so that you can select which tracks you would like to hear. Afterwords, you can select which hand to practice: left, right, or both. Alternatively, you can also just choose to watch and listen.

You may have noticed from the last image, but some midi files can have a crazy amount of notes, sometimes exceeding millions. Such large files are known as a "black midis." Their name comes from what the resulting music sheet with all the notes on it look like. A lot of black. These can be fun to watch or listen to.

Optimizations

A lot of optimizations went into making this game as time efficient as possible. I already mentioned that the file system was multi-threaded, but I have multi-threaded the loading of midi files as well. In addition to this, I have also created a separate faster file format (.fff). Midi files are optimized for storage, while mine is optimized for speed. In a midi file, messages of varying sizes are stored in any order, so they must be parsed message by message, and often even byte by byte. fff files, on the other hand, are read track by track, as all of the notes of each track can be read at once. From testing, larger midi files can take longer than 20 seconds to load, while the fff variation loads practically instantly. The only downside is that the file is quite a bit larger (58MB to 175MB).

Another important optimization is the implementation of a texture atlas. Sprite rendering cannot properly batch when several different textures are used. Every time a different texture is needed to be drawn, the batch is sent to the graphics card to be drawn and reset. Since each note consists of a rounded top sprite and a scalable middle sprite, the batch was interrupted with every note. To fix this problem, a texture atlas is used. This is when several textures are put into the same image file. When using these images, the engine must specify the region of the image to be drawn. After implementing this functionality, the difference was very noticeable when thousands of notes were being drawn. The drawing efficiency increased by over 1000%.

The last optimization I have been implementing is my own synthesizer. Since the default has so much latency, I thought it would be fun to try to make my own with minimal latency. So far, I can generate different waveforms at the correct frequencies. I am storing several one second buffers in memory so that they can be played with minimal latency. However, there is only a one note polyphony and this part is still a work in progress, so I don't recommend that you use it yet.

Download Link