# Ram consumption goes down after a while in VEPro



## Cookie Thumper (Oct 13, 2022)

Hi,

I hope you are well! I'm currently working on my orchestra setup, using a computer that has 192 GB of ram and on which I host various libraries in VEPro 7. 
I have been noticing the following for a while now: When I start the computer and then the session, the RAM is loaded up to a capacity of 92%. If this is important, my session is purged. After I run the session for a bit or just wait about half an hour, the ram load drops to about 30% and stays at that level even if I use the session normally.

I wonder why this happens and, on the other hand, whether I could optimise the footprint of my libraries to load even more samples, since only a fraction of the installed ram is apparently needed for operation. Because when I add more instruments to my session and the ram is fully loaded, the session just crashes. I use Kontakt and Sine on my computer.

Can anyone help me?


----------



## rgames (Oct 13, 2022)

Yeah... it does that.

I'm surprised it crashes. I've not done that in a while because I haven't needed to but I recall that the dump would happen sooner if you really pushed the RAM use.

Regarding why it happens, I've spent many years trying to understand how samplers use memory and it's still a mystery. You just try things and see what happens. I don't think anyone really understands what goes on other than maybe 5 people at Microsoft...!

But beyond that, is 192 GB really not enough? What kind of music are you writing that requires that much RAM, especially with everything purged?


----------



## youngpokie (Oct 13, 2022)

Cookie Thumper said:


> When I start the computer and then the session, the RAM is loaded up to a capacity of 92%. If this is important, my session is purged. After I run the session for a bit or just wait about half an hour, the ram load drops to about 30% and stays at that level even if I use the session normally.


It drops to 30% because VEP is physically accessing a fraction of the memory it claims to need. Windows is managing the total by moving the requested but yet unused amount to _committed_ memory and moving some of the physically used memory to either _cashed_ memory for high-speed access or to page file on the hard drive. You can observe the difference between used vs requested memory by each process in Task Manager - Details, if you enable Commit Size column.






If you don't want to wait for Windows to finish sorting it out, you can speed it up by using small apps like QuickCPU to "clean" memory and get to the 30% you're used to right away.

As far as I know, the maximum committable memory is limited by a factor of installed RAM and the size of the page file. So, if you want to load even more purged instruments into VEP, I think there are a few options to investigate:

- kill the processes with large commit sizes that you don't need - before you load VEP
- modify Kontakt settings to stream a larger share from disk than from memory
- review max voices in Kontakt (it reserves memory for every possible voice). For example, BWW sets max voices to 128 for Clarinet legato patch but I've never seen it use more than 90 on my system (EDIT: 102 with 3 mics active and playing very fast runs)
- experiment with increasing the size of the page file

Increasing the page file size can prevent memory crashes, but I've never heard of a way to control which processes get bounced to page file vs stay in RAM. You might be able to load more stuff at the cost of slower system overall because hard disk read times are always slower than RAM read times. However, you can always fix that by letting Windows auto-manage page file again. Since you have so much physical RAM, I'm actually curious how much disk space Windows is setting for your page file.


----------



## Cookie Thumper (Oct 13, 2022)

rgames said:


> Yeah... it does that.
> 
> I'm surprised it crashes. I've not done that in a while because I haven't needed to but I recall that the dump would happen sooner if you really pushed the RAM use.
> 
> ...


I should have been more specific: Kontact crashes. An error message appears that the system ram is low and then the session crashes.

I use products from the Berlin series of Orchestral Tools and use every patch in my template. At the same time, some microphone positions are loaded. Of course, I will never use every articulation in a project, but I like the approach of always having everything ready.



youngpokie said:


> It drops to 30% because VEP is physically accessing a fraction of the memory it claims to need. Windows is managing the total by moving the requested but yet unused amount to _committed_ memory and moving some of the physically used memory to either _cashed_ memory for high-speed access or to page file on the hard drive. You can observe the difference between used vs requested memory by each process in Task Manager - Details, if you enable Commit Size column.
> 
> 
> 
> ...


I have not had any experience with programmes like Quick CPU. I remember that I once cleaned the Ram with a batch file as you suggest to avoid the described spike and a crash of the session, if I understand you correctly. However, this made my session unstable as I recall. I will try the programme once, though.

I have changed the DFD value in the past but never noticed a change in the ram print after I rebooted.

The page file is flexible in size. I think I experimented with it too, however I did not maintain it then for a reason that was probably session stability. I would be happy to check how big my page file is.


----------



## rgames (Oct 13, 2022)

youngpokie said:


> It drops to 30% because VEP is physically accessing a fraction of the memory it claims to need.


Yeah - do you know why it does that sometimes and sometimes not? I've never been able to figure out what combination of commit/active/page file size/etc. makes it decide to dump. I've used templates of varying size over the years and some do the dump and some don't. I think I first started noticing it with EW Play libraries maybe 10 years ago. It's definitely not just a percentage of physical available, or physical+page. There's something more complicated than that.

Doesn't really impact anything in my case, I've just always been curious to know why that is.


----------



## Cookie Thumper (Oct 14, 2022)

rgames said:


> Yeah - do you know why it does that sometimes and sometimes not? I've never been able to figure out what combination of commit/active/page file size/etc. makes it decide to dump. I've used templates of varying size over the years and some do the dump and some don't. I think I first started noticing it with EW Play libraries maybe 10 years ago. It's definitely not just a percentage of physical available, or physical+page. There's something more complicated than that.
> 
> Doesn't really impact anything in my case, I've just always been curious to know why that is.


This is not the case with me: the ram footprint always goes down after some time in my case.

@youngpokie: My pagefile is 29.5 GB in size with the full session loaded.


----------



## jcrosby (Oct 14, 2022)

Sounds like it's your OS compressing files in memory. All operating system do this, in macos it's called memory pressure. It could very well be that VEP doesn't actually need all of the memory it initially reserves, and eventually Windows determines various files requesting memory can be compressed.









What Is Memory Compression in Windows 10?


Windows 10 uses memory compression to store more data in your system’s memory than it otherwise could. If you visit the Task Manager and look at your memory usage details, you’ll likely see that some of your memory is “compressed”. Here’s what that means.




www.howtogeek.com


----------



## Caleb Joshua (Oct 14, 2022)

I prefer to disable this function in windows because im paranoid about the sample data being in a state where its not ready to use. Start/type "services"/sysmain/properties/startup type:disabled/restart


----------



## xepocal (Oct 14, 2022)

Memory de/compression in Windows isn't like a .zip. It's many, many times faster.

>300 MiB/s per core to compress, >1 GiB/s per core to decompress on 10 year old machines.

Windows only compresses when it would've written to disk instead, so leaving memcomp on should be a win/win, regardless of how little or much RAM you have.


----------



## youngpokie (Oct 14, 2022)

rgames said:


> Yeah - do you know why it does that sometimes and sometimes not?


I have no idea how that process works in exact detail. However, simply loading instruments into a single Kontakt instance and watching the memory stats in Quick CPU is showing a few interesting things:






- The max amount of commit size is equal to the total amount of virtual memory available. 
- As I load instruments into Kontakt, they are loaded into physical memory 
(left green) and shown as committed size in virtual memory (right green bar). 
- "Cleaning" the memory via Quick CPU is showing some effect of compression that @jcrosby mentioned earlier - the commit size remained relatively stable, cash size is reduced, and physical in-use memory shows the largest gap (both green bars). However, I noticed that in Windows 11 I can no longer induce instant compression with Quick CPU reliably as before. Or perhaps it had to do with the version of Kontakt that had memory issues (version 6.2 IIRC?).
- When I am able to invoke the compression the difference in green bar sizes seems to always settle around 6 GB on my system and never exceed that. 






In the end, I can't see any way of forcing higher loads in VEP other than increasing the size of virtual memory available. The idea would be to increase the amount of committable memory available for VEP to claim (via page file, shutting down unneeded programs, whatever) and at the same time reasonably reducing its use of actual physical memory (judicious use of Kontakt voice counts, etc) and then taking advantage of compression. I wonder also if it is possible to load VEP instances gradually rather than all at once, to take advantage of compression.


----------



## jcrosby (Oct 14, 2022)

Caleb Joshua said:


> I prefer to disable this function in windows because im paranoid about the sample data being in a state where its not ready to use. Start/type "services"/sysmain/properties/startup type:disabled/restart


Literally no reason to do this. Your machine does, and will run better by having memory compression enabled.


----------

