Implementing pre-mixed layers in FMOD, and integrating

[Haakond]

Hi,

I needed to learn FMOD for a project I am working on, to implement music into a game. Learning the program was not that hard as I first thought, and this opens up a lot of possibilities for the soundtrack. And it certainly makes it easier to set a certain intro > loop > end in a more musical way than it would be for a programmer to do.

During my experiments with FMOD, I started wondering about two different things that maybe some of you know.

1. One of the advantages with a program like this is that you can use layers in the music, and for example throw in a choir when health is low. I do all composing, mixing and mastering in Logic and make a intro + loop + end file of each song. That means I bring in an already mastered track into FMOD. But if I want to separate different sections of the orchestra, does this means that I have to master each section by its own? Or would you bounce out each section into the mastering project, master it and then export each section again? If I have a low strings section playing an ostinato by itself and then fade in the rest of the strings into a different layer, the reverb from both sections would play on top of each other. And if you have many different layers, there would be a lot of extra reverb going on. Would this a problem?
I usually have front reverb for strings + piano, mid reverb for winds and horns, back reverb for rest of brass and perc, and a “glue” reverb for all the instruments

2.
Is it normal for composers to deal with the implementation in Unity/Unreal etc? Or is this a job for the devs? To me, it makes most sense that I don’t do anything in the game engine. But what do you guys do?

Thanks!

Click to read more...

 

[Billy Palmer]

Hey,

1) Regarding mastering stems - both approaches sound very sensible. I'd also like to know if one method is 'ideal'. Or what particular workflow other people have used (especially within logic).

Same re reverbs.

2) I haven't ever needed to delve into engines - the most involved I've ever had to get was simply setting up fmod events and parameters. On projects so far, I've been lucky enough to either work with a programmer or sound designer who have handled the engine and coding for fmod.

I feel it wouldn't hurt me to learn more about integration, but honestly there are other more important 'composer skills' that I want to develop first! I'm sure you've found this, but I feel that communication is way more important than the technical skills when it comes to audio integration.

Great topic btw

 

[Haakond]

Hey,

1) Regarding mastering stems - both approaches sound very sensible. I'd also like to know if one method is 'ideal'. Or what particular workflow other people have used (especially within logic).

Same re reverbs.

2) I haven't ever needed to delve into engines - the most involved I've ever had to get was simply setting up fmod events and parameters. On projects so far, I've been lucky enough to either work with a programmer or sound designer who have handled the engine and coding for fmod.

I feel it wouldn't hurt me to learn more about integration, but honestly there are other more important 'composer skills' that I want to develop first! I'm sure you've found this, but I feel that communication is way more important than the technical skills when it comes to audio integration.

Great topic btw

Thanks for the reply! I guess I have to do some experimenting, testing both solutions. And I think maybe I have to do some compromises. Maybe it is best to do it as easy as possible.

I feel so too. I think maybe the devs want the most control on things. But then again, if could be smart to take some work of their shoulders. Not sure what most composers do

 

[FredW]

Hey

1. Seems like you are overthinking this a bit, I don't know your background but how do you deliver stems in general? It's no difference than sending "mastered" stems to a dubstage, only difference is if you want that "glue" over all stems you might have to use FMOD's inbuilt plugins.

Print reverb fx etc to each stem you want to have separated. I don't really get that "extra" reverb thing?

2. I guess this is very personal and depends on the scale of the project, the game engine, your time and interest. Personally I have only worked on small indie projects and I do sound design, music and some programming. Not being able to implement my own music would be the same as not knowing how to sync my music to a video file. If I want to try out an idea it would be too time consuming to bother the programmer or sound designer every time. Being able to implement also puts you in control over your own work so less chance someone else screws it up. If this is your first game project though I understand that it could be a bit of a hassle to learn and be comfortable with in a short time.

 

[Henu]

You need to re-think the current worklow a bit differently if you're exporting stems for FMOD (or Wwise/any internal middleware).

First of all, "traditional" use of auxes is out of question unless you want to do all the exporting manually stem by stem (which is also an option, of course, just more cumbersome). If X needs to be exported without Y, you need to be sure that you're not hearing Y's reverb on the X track. Which means that unless you're exporting brass only, you can't share it's reverb with the woodwinds or you'll get the WW reverb on top of your brass stem. The routing basically needs to be pre-planned in advance and all the reverbs need to be assigned to the channels they are affecting. Yes, you could also use inserts but I prefer to do it this way for multiple reasons.
Here's how I recently routed mine (still WiP, haven't finished all mixing yet):




Secondly, concerning master bus FX, as you can see in the white "BUS" channel at the top mixer's right side, you can have all sorts of master FX for quick (preview) exports for the whole thing, but with the stems you need to make compromises. For example, that "glue" reverb I have in the BUS channel is impossible to do for them (also, while they are not bypassed in this particular picture, I always keep them bypassed until I will export the whole thing for a preview). For what it comes to bus compression and limiting- that is actually possible due to sidechaining as you can see from those black STEM -channels in the upper mixer. (Yes, it needs a ton of sends which you can't see from this picture.)

Keep also in mind, that when exporting in stems- whatever you want to put to the export bus channel, it will cumulate. So if you decide that you'll just "manually solo the tracks you want to hear in stem X because you want to hear that reverb, Ozone and whatever trueiron on your master bus", you'll be multiplying all that by the number of your stems. They will interact differently and will not be the same than the straight stem-exporting.

 

[Haakond]

You need to re-think the current worklow a bit differently if you're exporting stems for FMOD (or Wwise/any internal middleware).

First of all, "traditional" use of auxes is out of question unless you want to do all the exporting manually stem by stem (which is also an option, of course, just more cumbersome). If X needs to be exported without Y, you need to be sure that you're not hearing Y's reverb on the X track. Which means that unless you're exporting brass only, you can't share it's reverb with the woodwinds or you'll get the WW reverb on top of your brass stem. The routing basically needs to be pre-planned in advance and all the reverbs need to be assigned to the channels they are affecting. Yes, you could also use inserts but I prefer to do it this way for multiple reasons.
Here's how I recently routed mine (still WiP, haven't finished all mixing yet):




Secondly, concerning master bus FX, as you can see in the white "BUS" channel at the top mixer's right side, you can have all sorts of master FX for quick (preview) exports for the whole thing, but with the stems you need to make compromises. For example, that "glue" reverb I have in the BUS channel is impossible to do for them (also, while they are not bypassed in this particular picture, I always keep them bypassed until I will export the whole thing for a preview). For what it comes to bus compression and limiting- that is actually possible due to sidechaining as you can see from those black STEM -channels in the upper mixer. (Yes, it needs a ton of sends which you can't see from this picture.)

Keep also in mind, that when exporting in stems- whatever you want to put to the export bus channel, it will cumulate. So if you decide that you'll just "manually solo the tracks you want to hear in stem X because you want to hear that reverb, Ozone and whatever trueiron on your master bus", you'll be multiplying all that by the number of your stems. They will interact differently and will not be the same than the straight stem-exporting.

Thanks for the very in-depth answer. I had to read it multiple times to make sure I understood everything. I need to do some more experiments on my stuff, try to implement what you are doing

 

[Henu]

Sorry, I may have been too unclear on some points. Feel free to PM me (or ask here, just tag me so I don't miss it) if you want any clarification!

 

[Haakond]

Sorry, I may have been too unclear on some points. Feel free to PM me (or ask here, just tag me so I don't miss it) if you want any clarification!

Thanks for the offer, I think I have to send you a PM! Will get in touch with you shortly!

 

[Billy Palmer]

There was a question in the Spitfire mixing and mastering talk from today about mix buses for stems:

 

[Haakond]

There was a question in the Spitfire mixing and mastering talk from today about mix buses for stems:

Thanks for the heads up! I will check it out

 

[Richard Wilkinson]

Like library music, you need to avoid master processing & have effects on your group tracks, rather than using sends. For FMOD, setup your stems are you'll implement them - so instead of short strings, long strings, low percussion etc, you should have 'ambient', 'melodic accents' 'intensity level 2' etc

You need to plan out the implementation first, so you can set up your Logic sessions and stem organization in a way which will fit neatly into your FMOD or wwise session.

 

[Brasart]

Lots of great answers already, I'll add that I don't do a lot of master bus mastering when exporting stems for FMOD because it doesn't matter as much when in the context of a full game mix.

Be also wary of wanting to export too many individual stems; it can quickly eat a lot of storage space, and also CPU usage if you're doing vertical layering.
Ask your dev team now if/what ressources they want to allow for sound, or you might end up needing to redo a lot later on when optimization becomes a thing.

 

[pixel]

+1 to not using Master Bus effects. Groups/Busses for each stem/section you'll bounce to audio - this is where "mastering" should be done (meaning: finished sound).
It depends on the project what stems you'll need. Sometimes it's required to use different instrument sections to trigger them based on events in the game. Sometimes it's all about arrangement sections (then you bounce all sections together but you cut them based on arrangement: intro, chorus etc.)
It's highly recommended to use minimum real-time processing in FMOD. Every CPU cycle is precious in video games and unfortunately audio is at the end of the priority chain. Usually its build-in limiter/compressor. As the audio in games is away from loudness war and it follows modern standards, music level will be very low and mostly safe from clipping but it's good to tame peaks to save space for game sfx.

Implementation is done by sound engineer not composer. Sometimes it's designated person for implementation, very often it's just one person who do sound effects and is responsible for entire audio in the game. Normally sound engineer should know every detail about the game and audio needs and should communicate it with composer (what kind of stems are required etc.) Eventually it may be producer but then quality of communication depends highly on producer's ability to talk in music language I never heard about composer who would do audio implementation especially in game engine itself. Except if composer is also sound engineer who is hired to take care on entire project. But then salary is way different because in engine implementation (oreven in FMOD/Wwise) require extensive knowledge about engine's language (c#, c++ or sometimes just blueprints for UE4). And if it's job for Ubisoft then also knowledge about graphics 3D software is also "good to have"
So its huge list of skills going far beyond composer skill set (which is huge itself already of course ).