Templates: Sample purge make disable tracks & VE pro unnecessary?

JT3_Jon

Senior Member
I’m building my first real template, and in an effort to make it Ram / CPU efficient on both my main DAW and PC Slave, I had the idea of using VE pro but instead of focusing on only a few full kontakt instances, I’m loading multiple small kontakt instances with only a few patches in each, and then using the enable/disable buttons in VE pro, which can be automated to turn on/off & saved in my project files. This in theory would allow me to have a “kitchen sink” template and only enable the patches I needed for a particular project - i.e. a small chamber sound vs a large epic orchestra sound, etc, all loaded but all disabled by default, then enable them as necessary. Thus saving on resources to hopefully have sessions run more smoothly.

Then in my research I discovered the power of kontakts purge ability! I always assumed purge was for when you had already written most of your parts, you could then purge all the notes that you were not using to gain more RAM. I had no idea that it basically acts like an “on call” ability where you play a note and the sample is instantly loaded and plays back immediately. AMAZING!! So now this has me rethinking my entire template setup.
  1. Does having all your kontakt & EW Play instances loaded in a huge template in your DAW, but purged make more sense in practice than enabling/disabling tracks in VE pro? Seeing as how the audio ends up in your DAW anyway, I wonder if its better to cut out the middle man?
  2. Does this make using VE Pro on your DAW machine obsolete as you can now load all your kontakt instances in your DAW and have them purged until necessary? I wouldn’t imagine kontakt instances themselves use much, if any CPU, right?
  3. Are there any unforeseen issues for using the purge functions of Kontakt and Play?
I’ve run tests on my DAW machine in the past, and for whatever reason I have gotten MUCH better CPU performance hosting my VI’s in VE pro vs Cubase. I had a large project (progressive rock + orchestra & choir) and it was killing my Cubase performance, even at the highest buffer setting, however moving some processor heavy instruments into VE pro on the same machine made it so my Cubase VST performance was no longer peaking. No idea why. I've also ran test projects with both kontakt and omnisphere and was able to load many more instances of both in VE pro vs directly in Cubase. Perhaps its still best to run VE pro on my main DAW?

But either way I'm wondering if my whole idea of many small kontakt instances that can be disabled / enabled as needed is any more efficient than just purging the samples? For example I have 70 instances of kontakt for percussion alone, which could probably fit in 20 full instances, but then I would have to use kontakt busses for audio output processing which I dont have to in my small kontakt setup. Dont know the trade-off there.

Sorry for the rambling, but this is getting very confusing to figure out on my own. I’d LOVE to hear any / all feedback from you who have successful templates. Thanks in advance for your reply!
 

Grizzlymv

Active Member
I can't speak for vepro as I never succeeded in setting up a functional template with easy ways of enabling/disabling channel sets from my DAW. I did however tried a vepro template with purged instruments and it was killing my machine anyway. Even though the instance is technically empty, the instance itself uses resources. And after a certain amount loaded, my machine was running in troubles. So I did switch to a full instrument tracks with the disable/enable feature from the daw (cubase) and this was much better in terms of resources usage as when an instrument track is disabled, the instance is also unloaded from memory. So it's like if you deleted it.

With that setup j can have a template that have around a thousand tracks and I only enable those I need which don't overload the system.

I also got rid of vepro as it felt unnecessary in that setup.

Hope this help a bit. Good luck with your setup.
 

Guy Rowland

Senior Member
It's complicated...

Purged works brilliantly, and I've successfully used it now for a good 7 or 8 years I guess. However, it's not the ultimate silver bullet. Here's the two big things to look out for:

1. This only works on DFD streaming patches. If the Kontakt instrument loads samples into RAM, then purging won't work (or rather you'll play a note and nothing will happen until you play it again - not nice). Any instrument that can be tempo-synced would be an example of an instrument that can't be successfully purged, and sometimes it isn't obvious what these are. For example only yesterday I discovered some of the short patches in Swing More are not DFD.

2. The memory reported by Kontakt is not the total memory used by the instrument - this is just a readout of how much sample memory is used. Large GUIs and complex patches can take up a lot of memory even fully purged - I've seen some take well over 100mb. For example, my LASS template in VE Pro still takes up 6.5gb of RAM fully purged.

So particularly in the case of (1), disabled tracks would seem the way to go. Even there though, there can be hidden costs. While most disabled instruments take up very little RAM, some can be surprisingly hoggish. Yesterday (again) I found a major culprit that was causing my blank template to balloon out of control size-wise - Action Strikes. I had half a dozen instances disabled with different kits loaded, and they collectively took up 44mb of RAM. That may not sound like much, but that's added to every save and autosave, and the autosaves were getting ridiculously long and intrusive. I spent a morning pruning, and got my total template save from 130mb down to 18mb - that makes a colossal difference to saving times and the size needed for my dropbox project folder (I like to save many versions - it used to only take 8 versions and autosaves to use 1gb of RAM). The pruned instruments require yet another different approach, Cubase's Track Archives.

Track Archives are great in theory - you just load in a collection of tracks and away you go. It remembers the routing and all the settings, if there's a VE Pro instance associated it will automatically launch the right instance within VE Pro if the program is open, and of course the memory use in a project is zero until you actually need it. Being Steinberg however, there are problems - it can't remember folder tracks (which can be useful within a library to subdivide); it always dumps the new tracks at the end of a project not where you want and - by far the most annoying - it can be glitchy and not recall 100% accurately. I sometimes recall VE Pro tracks that load right in VE Pro, but don't connect to Cubase correctly. It doesn't take too long to sort it out, but it slows down the process which I strongly dislike.

Then there are multitimbral disabled tracks - you can have multiple midi and audio tracks for one instrument. Great, except these can be even more buggy than the Track Archives after a project is closed and re-opened, than a track re-enabled. Phantom duplicate outputs appear that you can't get rid of, colours and output orders get scrambled... it can be a right old mess.

Finally there are Track Presets. These are neat and work from a tag-based selector. These also are not the dream ticket however - no routing or aux send info is retained, so it's not much use for when spatialisation is required, and annoyingly unlike adding a regular new track you can't select the right output group from the initial dialog.

So pick the bones out of that lot. Much as I don't want to, I've ended up using every one of these techniques in the same template, in order to work round the glitches, inefficiencies and missing functionality.
 
OP
JT3_Jon

JT3_Jon

Senior Member
I can't speak for vepro as I never succeeded in setting up a functional template with easy ways of enabling/disabling channel sets from my DAW. I did however tried a vepro template with purged instruments and it was killing my machine anyway. Even though the instance is technically empty, the instance itself uses resources. And after a certain amount loaded, my machine was running in troubles. So I did switch to a full instrument tracks with the disable/enable feature from the daw (cubase) and this was much better in terms of resources usage as when an instrument track is disabled, the instance is also unloaded from memory. So it's like if you deleted it.

With that setup j can have a template that have around a thousand tracks and I only enable those I need which don't overload the system.

I also got rid of vepro as it felt unnecessary in that setup.

Hope this help a bit. Good luck with your setup.
Thank you for sharing your thoughts. Do you have a large database or have a large "quick load" in Kontakt. Find it very surprising that purged kontakts were causing your system issues! I have disabled the database in my kontakt (never use it to find sounds anyway) so hopefully I wont run into the same issues!!

Thank you very much for sharing though, much appreciated.
 

Grizzlymv

Active Member
I have cleaned the database so I don't have anything there. I don't use the quick load at all. I use one instance of kontakt per instrument track, which contains only one articulation. Therefore I don't need to have the DB. Depending of your system specs you might be fine. Hopefully you'll find the right balance for the gears you have. On my side, I wouldn't return to a vep setup. Disabled instrument tracks is so easy to work with. I just love it. But that just works in cubase so if you have another daw then you're screwed unless it support similar functionality.
 
OP
JT3_Jon

JT3_Jon

Senior Member
Helly Guy! Hope things are going well with you and thank you very much for your detailed post! You have opened my eyes to options I previously had not considered, though its a real shame each one comes with its own set of problems. :(

It's complicated...

Purged works brilliantly, and I've successfully used it now for a good 7 or 8 years I guess. However, it's not the ultimate silver bullet. Here's the two big things to look out for:

1. This only works on DFD streaming patches. If the Kontakt instrument loads samples into RAM, then purging won't work (or rather you'll play a note and nothing will happen until you play it again - not nice). Any instrument that can be tempo-synced would be an example of an instrument that can't be successfully purged, and sometimes it isn't obvious what these are. For example only yesterday I discovered some of the short patches in Swing More are not DFD.

2. The memory reported by Kontakt is not the total memory used by the instrument - this is just a readout of how much sample memory is used. Large GUIs and complex patches can take up a lot of memory even fully purged - I've seen some take well over 100mb. For example, my LASS template in VE Pro still takes up 6.5gb of RAM fully purged.
Thank you - wasn't aware of these limitations. BTW, I find it amazing that LASS takes up that much RAM fully purged!! WOW!!

So particularly in the case of (1), disabled tracks would seem the way to go. Even there though, there can be hidden costs. While most disabled instruments take up very little RAM, some can be surprisingly hoggish. Yesterday (again) I found a major culprit that was causing my blank template to balloon out of control size-wise - Action Strikes. I had half a dozen instances disabled with different kits loaded, and they collectively took up 44mb of RAM. That may not sound like much, but that's added to every save and autosave, and the autosaves were getting ridiculously long and intrusive.
Where are you disabling your tracks? Is this inside VE pro or in Cubase? If in VE pro, I take it you have them coupled to your Cubase project? I tend to keep my decoupled for this very reason, though I have in a haste forgotten to save after adding instruments and was sad when I reopened the project the next day. :(

The pruned instruments require yet another different approach, Cubase's Track Archives.Track Archives are great in theory - you just load in a collection of tracks and away you go. It remembers the routing and all the settings, if there's a VE Pro instance associated it will automatically launch the right instance within VE Pro if the program is open, and of course the memory use in a project is zero until you actually need it. Being Steinberg however, there are problems - it can't remember folder tracks (which can be useful within a library to subdivide); it always dumps the new tracks at the end of a project not where you want and - by far the most annoying - it can be glitchy and not recall 100% accurately. I sometimes recall VE Pro tracks that load right in VE Pro, but don't connect to Cubase correctly. It doesn't take too long to sort it out, but it slows down the process which I strongly dislike.
Just ran some quick tests with Track Archives and it appears that they only work when loading VE pro as an instrument track and not a rack instrument. I personally prefer to load VE pro as a rack instrument as I can move the individual outputs underneath each midi track for that instrument (i.e. my flute return can be put in my flute folder) - makes it easier for me to automate volume & insert effects. I take it you must be using Instrument tracks for these track archives? If so, do you like that all audio returns for that instrument are in lanes on a single instrument track, rather than individual tracks? Or perhaps you dont use multiple return inputs? And yes, all the other problems (folder tracks are not saved/recalled, midi tracks loosing routing when loading, etc) make this a far from perfect solution. Have you notified Steinberg about these issues? If so I'd love to join you in a crusade to get bugs like these fixed, though given Steinberg's track record of fixing bugs, I'm not going to hold my breath.

Thanks again for your awesome post! You have given me much to think about and to try. Would love further information if possible.
 
OP
JT3_Jon

JT3_Jon

Senior Member
I have cleaned the database so I don't have anything there. I don't use the quick load at all. I use one instance of kontakt per instrument track, which contains only one articulation. Therefore I don't need to have the DB. Depending of your system specs you might be fine. Hopefully you'll find the right balance for the gears you have. On my side, I wouldn't return to a vep setup. Disabled instrument tracks is so easy to work with. I just love it. But that just works in cubase so if you have another daw then you're screwed unless it support similar functionality.
Wow! you too have given me things to consider. In the past I have run into resources maxing out in Cubase, and to solve it I simply moved more instruments into VE-pro on my host machine instead of in my DAW, and that for some reason solved my problem and I could continue the project. So I'm a little weary of having too much loaded in Cubase, but maybe I can find a compromise for less used instruments being available as disabled instruments and/or instrument preset tracks. But I do see the benefits of having everything available in a daw and enable as needed. Will have to try things and see how it goes. Thanks again for sharing your thoughts and experiences! Really appreciate it!
 

Grizzlymv

Active Member
Just before I went all instrument tracks, I was still keeping all my strings, woods and brass in vep and kept the rest in the daw. Was working ok. But I have better performances with everything in the daw now as I have less tracks active at the same time (only those I need). I'm sure vep would do a better job but given the fact I can't be as granular as how I'm set up right now with just the daw. The day I can control vep from within cubase as easily as enabling/disabling an instrument track in cubase I might reconsider but for now I like it a lot how it works.
 

Guy Rowland

Senior Member
Where are you disabling your tracks? Is this inside VE pro or in Cubase? If in VE pro, I take it you have them coupled to your Cubase project? I tend to keep my decoupled for this very reason, though I have in a haste forgotten to save after adding instruments and was sad when I reopened the project the next day. :(
Hi Jon - all disabling in Cubase, and yes these are coupled to a VE Pro instance. I've not gone down the VE Pro disabled road, because I can't figure out a seamless background way of enabling - in theory automation can do this, but it seems fiddly. I'm hoping that VE Pro 7 might have Wake On Midi - what you need is a totally disabled VE Pro template, then instruments wake as they receive a midi note. Here's hoping, but until then it feels like another layer of complexity, and it's pretty complex as it is for me.

Just ran some quick tests with Track Archives and it appears that they only work when loading VE pro as an instrument track and not a rack instrument. I personally prefer to load VE pro as a rack instrument as I can move the individual outputs underneath each midi track for that instrument (i.e. my flute return can be put in my flute folder) - makes it easier for me to automate volume & insert effects. I take it you must be using Instrument tracks for these track archives? If so, do you like that all audio returns for that instrument are in lanes on a single instrument track, rather than individual tracks? Or perhaps you dont use multiple return inputs? And yes, all the other problems (folder tracks are not saved/recalled, midi tracks loosing routing when loading, etc) make this a far from perfect solution. Have you notified Steinberg about these issues? If so I'd love to join you in a crusade to get bugs like these fixed, though given Steinberg's track record of fixing bugs, I'm not going to hold my breath.
You're absolutely correct - I've switched to instrument tracks from VE Pro for this very reason. I then add as many audio outputs as I need (right click the instrument icon in the inspector of Cubase). Really, rack instruments are arguably redundant now as track instruments can have multiple midi track assigned and multiple outs. The only pain is that I tend to skip midi channel 1 and output 1 with this way of working as midi and audio are combined on track 1, whereas with all other tracks and channels they are separate, and that does my head in. So the main instrument track I pretty much treat as a folder track now!

I'd be delighted if you joined my crusade against the bugs, go here - https://www.steinberg.net/forums/viewtopic.php?f=253&t=123873&sid=4e52f9f7765d9f056992c9f49a90f6f4 , I've also added a support request recently on this issue, but so far have heard nothing.
 
OP
JT3_Jon

JT3_Jon

Senior Member
You're absolutely correct - I've switched to instrument tracks from VE Pro for this very reason. I then add as many audio outputs as I need (right click the instrument icon in the inspector of Cubase). Really, rack instruments are arguably redundant now as track instruments can have multiple midi track assigned and multiple outs. The only pain is that I tend to skip midi channel 1 and output 1 with this way of working as midi and audio are combined on track 1, whereas with all other tracks and channels they are separate, and that does my head in. So the main instrument track I pretty much treat as a folder track now!
Wait, can I get some clarification on this. When I use an instrument track, all return audio ports (not just channel 1 & output 1) are associated with that instrument track and cannot be moved from the "automation lanes." See picture:

Cubase Instrument.png

So in the picture you see I have an instance of Omnisphere coming in from VE pro with each omnisphere part on a seperate audio input. My brain, at least in instances like this where each sound is serving a different purpose, likes to have the audio return under the midi track, which you can do with Rack instruments (see picture below)

Cubase Rack.png

Is it possible to organize instrument tracks like the above picture? I guess I could create new groups for each instrument but that seems like a big waste of resources. Maybe you know of a way to break the audio returns out of the instrument tracks?

I think I'm for sure going to use instrument tracks for my orchestral instruments though, since I like to bus them to groups in VE pro anyway and in all honesty dont do much volume automation (try to take care of this in the midi as much as possible) but for synths like this where I tend to use a lot of insert effects and automation of said inserts, I'm torn on what to do.
 
Last edited:

Guy Rowland

Senior Member
Ah I see - yes AFAIK that's a limitation of Instrument Tracks. I've always tended to bunch mine together anyway and have, say, all the LASS midi followed by the LASS audio, so I'm not too bothered, but i see it will be more frustrating for you if you want to combine them both.
 

Bender-offender

Active Member
From what I've read, the number of VEP Outputs enabled for each VEP instrument in Cubase will make your CPU work harder on whichever computer has VEP on it (slave and/or master). I'm not sure how accurate this is, but it may be something to consider. That and the whole "more instruments in a single VEP instance vs more VEP instances with less instruments" still boggles my mind. Sometimes I've considered just going back to my old and reliable JV-1010. ;)
 

Mihkel Zilmer

Senior Member
From what I've read, the number of VEP Outputs enabled for each VEP instrument in Cubase will make your CPU work harder on whichever computer has VEP on it (slave and/or master). I'm not sure how accurate this is, but it may be something to consider. That and the whole "more instruments in a single VEP instance vs more VEP instances with less instruments" still boggles my mind. Sometimes I've considered just going back to my old and reliable JV-1010. ;)
I have tested various configurations on Win10, latest Cubase & VEPro, 1 master + 1 slave.

- First approx. 100 audio returns in any VEPro instance have very little impact on performance
- Somewhere above 100 audio returns per instance the performance takes a sudden, serious hit
- I keep my VEPro instances at 64 stereo returns (total of 640 audio return tracks from 10 instances)
- If I turn all 640 audio returns off the ASIO meter barely moves and I see only a tiny increase in performance

On a separate note, the number of active MIDI ports seems to make a difference for performance too - I have 8 ports active with almost no effect on performance, but activating 16 ports per instance comes with a noticeable performance drop.
 
OP
JT3_Jon

JT3_Jon

Senior Member
I have tested various configurations on Win10, latest Cubase & VEPro, 1 master + 1 slave.

- First approx. 100 audio returns in any VEPro instance have very little impact on performance
- Somewhere above 100 audio returns per instance the performance takes a sudden, serious hit
- I keep my VEPro instances at 64 stereo returns (total of 640 audio return tracks from 10 instances)
- If I turn all 640 audio returns off the ASIO meter barely moves and I see only a tiny increase in performance

On a separate note, the number of active MIDI ports seems to make a difference for performance too - I have 8 ports active with almost no effect on performance, but activating 16 ports per instance comes with a noticeable performance drop.
Thank you so much for sharing your experiences! This is extremely helpful! Thank you!!
 

benatural

Active Member
I have tested various configurations on Win10, latest Cubase & VEPro, 1 master + 1 slave.

- First approx. 100 audio returns in any VEPro instance have very little impact on performance
- Somewhere above 100 audio returns per instance the performance takes a sudden, serious hit
- I keep my VEPro instances at 64 stereo returns (total of 640 audio return tracks from 10 instances)
- If I turn all 640 audio returns off the ASIO meter barely moves and I see only a tiny increase in performance

On a separate note, the number of active MIDI ports seems to make a difference for performance too - I have 8 ports active with almost no effect on performance, but activating 16 ports per instance comes with a noticeable performance drop.
This is really interesting. I'm curious, what buffer size are you working with when you have 100 returns per instance? Also do you get pops and clicks with this amount? Also curious about your system specs and the speed of your network.

Thanks for sharing this info!
 

JohnG

Senior Member
Skimming through this discussion makes my head hurt.

Long ago (five years?) a really fast PC slave computer cost a lot -- as much as $5k if you really went crazy with PCIe SSDs and all that. Today, a PC that is much better costs less than $2k. And VE Pro has improved a great deal too since then.

Given the lower cost, I vote for avoiding complexity of all the purgings, fiddlings, and so on, and am all in for BRUTE FORCE. More PCs!
 

benatural

Active Member
Skimming through this discussion makes my head hurt.

Long ago (five years?) a really fast PC slave computer cost a lot -- as much as $5k if you really went crazy with PCIe SSDs and all that. Today, a PC that is much better costs less than $2k. And VE Pro has improved a great deal too since then.

Given the lower cost, I vote for avoiding complexity of all the purgings, fiddlings, and so on, and am all in for BRUTE FORCE. More PCs!
All true! Thing is the master is always going to be a bottleneck no matter how many slaves you have. If I understand my facts correctly, slaves get you lower latency, more loaded at once, distributed cpu overhead at the slave level (all good things btw) but they don't increase the amount of simultaneous connected VEP instances on the master or the amount of data it can recieve/process in real time. Is that inaccurate?
 

Mihkel Zilmer

Senior Member
This is really interesting. I'm curious, what buffer size are you working with when you have 100 returns per instance? Also do you get pops and clicks with this amount? Also curious about your system specs and the speed of your network.

Thanks for sharing this info!
You're welcome!

The master is an i7 6900k slightly overclocked, the slave is an older 4820k overclocked to 4.3GHz.
The master runs Cubase and VEPro with Strings & Brass. Woodwinds, percussion, miscellaneous and ethnic instruments etc. are hosted on the slave. I'm using Gigabit ethernet - I can't saturate its bandwidth with only one slave - I tested that and the slave runs out of resources before that happens. In a situation with 2+ slaves you could hit the bandwidth cap though, and may want to consider 10Gb.

For lighter projects (less than say 10 instruments playing back) I can work with the buffer at 128.
For orchestral work (usually 30+ instruments), the buffer stays at 256 for almost the entire time and only gets bumped up to 512 when I bring in the final, more intensive mastering plugins (izotope, Kush etc, all of which introduce their own latency anyway, so I don't use them while composing). No clicks, no pops, though that was not the case when I first built this machine - some drivers were causing issues. VEPro buffer multiplier set to 1.

All true! Thing is the master is always going to be a bottleneck no matter how many slaves you have. If I understand my facts correctly, slaves get you lower latency, more loaded at once, distributed cpu overhead at the slave level (all good things btw) but they don't increase the amount of simultaneous connected VEP instances on the master or the amount of data it can recieve/process in real time. Is that inaccurate?
Technically it's accurate, but a single master PC can handle A LOT. I used to have 1 master + 3 slaves. You can have thousands of returns and still have enough juice left over to run all your plugins too, especially if you offload all the samples to slaves.

After VEPro 6 was released and the disable feature introduced I now keep some, far more rarely needed parts of my template disabled in VEPro and could get rid of 2 slave PCs with no performance drop, just a short wait to re-enable the rare instruments..
 

JohnG

Senior Member
Thing is the master is always going to be a bottleneck no matter how many slaves you have. If I understand my facts correctly, slaves get you lower latency, more loaded at once, distributed cpu overhead at the slave level (all good things btw) but they don't increase the amount of simultaneous connected VEP instances on the master or the amount of data it can recieve/process in real time. Is that inaccurate?
Honestly -- I'm not sure I understand the question, though the reason I don't understand you could be the way I'm set up. I use VE Pro in standalone and have hardware for audio -- MidiOverLAN CP for midi. Audio goes straight into Pro Tools.

Consequently, my "master" holds Digital Performer and a number of virtual instruments (especially the ones that require precise tempo sync). Only on that one computer do I use VE Pro in server mode.

But briefly, if you have a computer that is solely dedicated to brass, for example, it can run just about anything you throw at it. For $1,500 or $1,800 per section, you can forget about mechanics and focus on writing.