What the Flip is Non-Linear Music?
We shared a short video (see above, apologies for the crappy resolution) earlier in the week of our audio test scene. We’re in the process of integrating FMOD into our game, which is a piece of audio middle-ware. This essentially bolts onto Unity (the game engine) and gives us enhanced audio capabilities.
One of the key things it gives us is the ability to utilise non-linear music.
We thought we’d take a moment to explain what non-linear music is, and why we’re super excited to have it in our ‘bag of storytelling tricks’ for Röki.
We’re going to be breaking this down assuming no prior knowledge and that you’re not an audio super-buff. If you are an audio wizard, apologies, you’ll already be very familiar with the content of the rest of the post. It’s also worth pointing out that this post is not written by an audio wizard but a muggle developer so hopefully it’ll give an accessible explanation and insight into to what non-linear music is. If you’re a games player you may find this an interesting insight to what’s going on under the hood in some of your favourite games!
Linear Music
Before we dig into its non-linear counterpart we should first look at linear music and what that is. Linear music is a complete music track that starts playing at the start of the piece and plays until it reaches the end.
Example of non-linear music are every song you hear on the radio, or any film score.
In games you have some more capabilities with linear music. You can:-
make it loop or repeat
Change the volume of the whole track, including fading it out
The Two Aspects of Non-Linear Music
Broadly speaking there are two ways in which non-linear music can be achieved. These two aspects can be described as:-
Vertical Mixing
Horizontal Mixing
It’s important to note that these two aspects are not mutually exclusive, but more on that later.
Non-Linear music - Vertical Mixing
In order for vertical mixing to work your music track must be separated into a number of ‘stems‘. These ‘stems’ are the component parts of the track. For example, you could break down a track into it’s component drum, bass, piano and flute layers, as seen below.
This then gives you the ability at any time to fade in or out the different component part of a track. This can be used to make the game feel more reactive. For example, if a dangerous event occurs then the drum layer may fade in, when the danger has passed it fades out again. This is vertical mixing.
Non-Linear music - Horizontal Mixing
The other aspect to non-linear music is horizontal mixing. This is when one long track is divided up into smaller sections. The first section plays, but when it approaches the its end, rather than blindly going onto the next section in a linear fashion it makes a choice!
It can pick from the different sections you have. This choice could be based on a random variable or a game-play factor, such as a story event or puzzle state.
This gives your music much more variety, allows it to be intelligent and react as the events unfold on screen. _For example, choosing to transition to a new upbeat section can add to a player's sense of accomplishment. At the very least it can get you a lot more mileage from your track, rather than a linear track that is sequentially the same each time, it puts the controls for the emotional power of the music in your hands!
It is important to note that in order to use horizontal mixing these sections must be written so they they can each transition into each other so require some thought as part of the composition process.
BOTH
As mentioned earlier, these two aspects are not mutually exclusive. Within each horizontal music section you can also have different vertical music layers that you can drop in or out.
So next time you’re playing game and you notice the music react, chances are it’s down to the use of non-linear music. Anyhow, we’ll leave it there for this week. More info on what we’re doing on the audio side on the coming weeks. Apologies to any audio buff appalled by my layman’s explanations ;)
Cheers,
Alex