# Batch resave doesn't "last"



## [email protected] (Sep 3, 2020)

Hey, all,

I just bought the Venice Modern Strings in the latest Fluffy sale and it took 60 minutes to load all five instruments. So I did a batch re-save and suddenly it took only half a minute to load an instrument. But today it's the same all over again. So how do I make a batch re-save "last" so that the loading time stays fast after I turned the computer off and on again?


----------



## Manuel Stumpf (Sep 3, 2020)

I think it has nothing to do with the batch resave.

Fact:
When a library has been loaded, then unloaded and you then load it again, that second time it will load faster than the first time (as long as you havn't rebooted your computer).

First of all it could be virus scanners delaying the first time you load the library, but not the second time.
Second it might have something to do with the operating system already having some files mapped to memory, whatever.
This advantage goes away after rebooting your computer.

Do you stream Venice Modern Strings from an SSD? 60 minutes seems to be a lot.


----------



## labornvain (Sep 3, 2020)

I'm very interested in this topic as well.


----------



## Manuel Stumpf (Sep 3, 2020)

That problem is not related to Venice Modern Strings alone.

I have made the following observations:
Libraries which consist of ten thousands or even hundred thousands of individual sample files load much slower than sample libraries having the samples collected in a view gigabyte sized files.
All libraries being structured like this load slower.

I think I never saw a non-Kontakt Player library having these huge sample files.
It might as well be that the option of having many samples collected in a single big file is only available for the developers who pay NI for encoding their library for Kontakt Player.


----------



## EvilDragon (Sep 3, 2020)

It's normal. Batch resave basically loads each NKI at least once, so next time you load it it's gonna be faster because those files are now cached. After a computer restart, everyting is back to how it was.


----------



## Kony (Sep 3, 2020)

60 minutes can't be normal though.


----------



## RonOrchComp (Sep 3, 2020)

EvilDragon said:


> It's normal. Batch resave basically loads each NKI at least once, so next time you load it it's gonna be faster because those files are now cached. After a computer restart, everyting is back to how it was.



With much love and respect EDrag 

That doesn't sound right. Unless I am misunderstanding. When I do a batch resave, it is forever


----------



## Thunderous Light (Sep 3, 2020)

EvilDragon said:


> It's normal. Batch resave basically loads each NKI at least once, so next time you load it it's gonna be faster because those files are now cached. After a computer restart, everyting is back to how it was.



Wait, so then, every time we reboot, or just restart our computer after powering it down, our batch saves are gone? If that is true, so then are batch saves just intended for the moment you are going to be using a specific library?


----------



## bcarwell (Sep 3, 2020)

I have heard this too that restarts destroy what batch resave accomplished. But if true it doesn't seem worth it to me to have to endure a resave every time after a restart and only for time saving benefits during that session before shutting down. What's the point ?


----------



## RonOrchComp (Sep 3, 2020)

Restarts DO NOT destroy what batch resave has accomplished here. Never has.


----------



## davidson (Sep 3, 2020)

So what does a batch re-save actually do?


----------



## Manuel Stumpf (Sep 4, 2020)

The problem mentioned has absolutely nothing to do with batch resave.

It is just that loading a library a second time is faster than the first time, because something is already cached.

The only thing is: 60 minutes seems to be too much
I suspect either it is not on an SSD (it should be, otherwise it is a problem if you load a hundred thousand sample files) or it has to do something with virus scanner.


----------



## Germain B (Sep 4, 2020)

I had, too, some instruments (nki files) that needed few minutes to load each time although I did several batch-resave.
The problem, as *@Manuel Stumpf *said, came from Windows Defender, scanning some of those files. Putting the entire samples' folder (the whole disk actually) in an exception list of the antivirus solved the problem. And I actually disabled everything I could in the security parameters of Windows as my computer is never connected to internet. (But I'm not sure I would recommend to do that...)


----------



## Paul Jelfs (Sep 4, 2020)

I tend to find NON KONTAKT PLAYER libraries are really guilty of being slow loaders after restarting. 
For some reason , my Berlin Strings Expansion takes about a minute or more to load each patch from SSD. 

Batch save works well until next time i load the project etc. 

So are you saying that most sample libraries should be set as exempt from Windows Defender? 
Will try this


----------



## Paul Jelfs (Sep 4, 2020)

Next question how do you do this in Windows Defender ! ? I know how to add rules to stop scanning certain software or ports - is it the same for a disk ? Do you change both in bound or out bound ?


----------



## Germain B (Sep 4, 2020)

Paul Jelfs said:


> Next question how do you do this in Windows Defender ! ? I know how to add rules to stop scanning certain software or ports - is it the same for a disk ? Do you change both in bound or out bound ?


I'm not in front of my Windows computer but I think it was this method : https://support.microsoft.com/en-us/help/4028485/windows-10-add-an-exclusion-to-windows-security

Edit : More details on this link : https://winaero.com/blog/exclusions-windows-defender-windows-10/


----------



## Germain B (Sep 4, 2020)

Paul Jelfs said:


> For some reason , my Berlin Strings Expansion takes about a minute or more to load each patch from SSD.
> 
> Batch save works well until next time i load the project etc.
> 
> So are you saying that most sample libraries should be set as exempt from Windows Defender?


Yes, I was having this exact problem with the same library until the Windows Defender' solution.

If you open the Task manager during the loading of the library you should see that the anti-malware process (can't recall the exact name) is pretty active.


----------



## EvilDragon (Sep 4, 2020)

RonOrchComp said:


> With much love and respect EDrag
> 
> That doesn't sound right. Unless I am misunderstanding. When I do a batch resave, it is forever



*As far as fixing incorrect sample paths*, yes, that is forever (or until something is moved somewhere else and you need to do it again). *As for speed of loading*, no, it's not forever, as soon as you restart you're back to initial speed of loading as if you're loading the library for the first time after a cold boot. *Except in case of large pre-K4.1 libraries *which will load faster always due to more efficient binary format that was introduced in K4.2 and above. This is forever. But for anything created for K4.2 and newer, there are no loading speed benefits in batch resaving.


----------



## MartinH. (Sep 4, 2020)

EvilDragon said:


> But for anything created for K4.2 and newer, there are no loading speed benefits in batch resaving.



Then why do people do it at all? I always thought this is mainly for loading speed increases, but I never felt a need to try it.


----------



## zvenx (Sep 4, 2020)

EvilDragon said:


> *.....* But for anything created for K4.2 and newer, there are no loading speed benefits in batch resaving.



I learn something new everyday. Did not know this.
rsp


----------



## Manuel Stumpf (Sep 4, 2020)

My experience is totally different.
A batch resaved library loads much quicker than a not batch resaved one.
This speed increase stays across any reboots (especially on spinning disks and libraries with many sample files this is substantial).
(Speed increase might be due to the fact that Kontakt does not need to search sample files after a batch resave??).

But still the first loading after a reboot will take longer, than subsequent loads.
Which is a general phenomenon not related to batch resaving. You can't do anything about it.

Numeric example:
- not batch resaved library load: 10 minutes
- batch resaved library load: 1minute
- subsequent load of batch resaved lib: 10 seconds

In this example the first speed increase from 10 minutes to 1 minute is permanent.
The second speed increase from 1 minute to 10 seconds is not.

Edit:
Batch resave might indeed not decrease loading time for libraries that use huge monolithic sample files, where the samples are collected in a view big files. I have no proof in that case.


----------



## EvilDragon (Sep 4, 2020)

Again it really depends how old a library is, how big the NKI files are (consequently how many zones are in there), and so on. It's not a simple situation or a one-size-fits-all thing.

Here's a blog post from Blake that pretty much explains everything.









The Blake Robinson Synthetic Orchestra - Tutorials » Batch Re-saving


Synthetic Orchestra. Now up to 736 pieces of music. That's 26 hours, 11 minutes of listening!




www.syntheticorchestra.com


----------



## labornvain (Sep 4, 2020)

EvilDragon said:


> Again it really depends how old a library is, how big the NKI files are (consequently how many zones are in there), and so on. It's not a simple situation or a one-size-fits-all thing.
> 
> Here's a blog post from Blake that pretty much explains everything.
> 
> ...


So, according to Blake, who seems very knowledgeable, batch resaves are permanent. The re-save process rewrites the NKC file which is stored on your hard drive and is not affected by a restart

So why are some people claiming that rebooting your computer wipes the cache file and therefore your batch resave along with it?

In my personal experience, I can't say that I've ever noticed a library that I have batch resaved reverting to its pre batch resaved state.

I do occasionally come across libraries that appear to have not been batch re-saved.

But I always assumed that is because I just missed that one, probably because the whole idea of having to batch re-save is so fucking stupid, and such a complete creativity killer when it happens that I occasionally find myself procrastinating doing it.

The weakest link in Kontact has always been the tediousness and time devouring process of navigating to a patch and then loading it in it.

So time consuming, and inspiration killing is this process that it's forced people to build giant multi gigabyte templates that require hot noisy massively energy-consuming slave computers just to avoid it.

I don't mean to sound unsympathetic. I fully understand the challenges imposed buy a Hurdy Gurdy patch that is 7 GB and has 250000 samples.

But it does seem that in this day and age, and with the amount of money involved in these libraries, that it shouldn't be too much trouble to have them batch re-saved in advance.

Oh yeah, I almost forgot. If there is a process that wipes the NKC file, would it be possible to lock that file, make it read only, so that it truly is permanent. Just a thought.

Now I'm going to go in and cmod all my NKC files.


----------



## [email protected] (Sep 5, 2020)

Hi, all - WOW! I didn't expect that much interest in this thread. So thanks for all you replies. Since I am a hobbyist I didn't manage to read through this thread before the weekend. 
I load Venice Modern Strings from an external SSD which has a lot of other libraries which don't cause this problem (for example Dominus Pro, Strezov Choirs and Ark 2). I had this trouble only with 8dio products. So I will try the Antivirus-exception-method.

Thanks a lot again!


----------



## EvilDragon (Sep 5, 2020)

labornvain said:


> The re-save process rewrites the NKC file which is stored on your hard drive and is not affected by a restart



NKC file is only relevant to Kontakt Player libraries.



labornvain said:


> So why are some people claiming that rebooting your computer wipes the cache file



Because I'm not talking about the NKC cache file, but operating system's internal file access cache. That one DOES get cleared when you shut down.


----------



## GMusic (Sep 13, 2020)

I'm having this issue with Anthology from 8DIO. I batched resaved last week and the empty Anthology instrument loaded within seconds. But today I tried again and it took over 3 minutes to load an EMPTY instrument. What is going on?


----------



## EvilDragon (Sep 14, 2020)

If you turned off your computer in the mean time, that would likely be the reason.


----------



## José Herring (Sep 14, 2020)

Germain B said:


> I had, too, some instruments (nki files) that needed few minutes to load each time although I did several batch-resave.
> The problem, as *@Manuel Stumpf *said, came from Windows Defender, scanning some of those files. Putting the entire samples' folder (the whole disk actually) in an exception list of the antivirus solved the problem. And I actually disabled everything I could in the security parameters of Windows as my computer is never connected to internet. (But I'm not sure I would recommend to do that...)


Thank you for this. It worked like a charm. 

I was getting really slow load speeds. I had switched to NVME and SSD for sample streaming and was shocked that it didn't increase load times too much. Batched resaved and like the original poster thought it worked but then rebooted my computer and back to long long times.

Made changes to Defender like you suggested and now I'm loading about 30gigs of samples in less than 1 minute. That's what I'm talking about!!!


----------



## Kent (Sep 14, 2020)

José Herring said:


> Made changes to Defender like you suggested and now I'm loading about 30gigs of samples in less than 1 minute. That's what I'm talking about!!!


----------



## b_elliott (Sep 14, 2020)

I did the "exclusion process" per the @Germain B provided links. 

My problem library load has been L & S Strings taking more than 5 minutes to load inside Kontakt 6. 

Doing the Windows Security exclusion makes a big difference for me. Load times now 15 or so seconds so far. Thank you Germain!

System specs:
WINDOWS10
INTEL i7-4770 @3.40GHz
1TB ssd Crucial MX500; 24GB RAM

EDIT: Silly me, I missed seeing @José Herring 's above post, so this kinda echos him.


----------



## Dr.Quest (Sep 14, 2020)

MartinH. said:


> Then why do people do it at all? I always thought this is mainly for loading speed increases, but I never felt a need to try it.


I batch re-save when the library can’t find the samples. There are some that I get from smaller companies that have the sample path screwed up and a batch re-save fixes it. If there are a lot of NKI files they all load fine then. Main reason for me. If the library loads fine to begin with I don’t bother.


----------



## GMusic (Sep 25, 2020)

Germain B said:


> I'm not in front of my Windows computer but I think it was this method : https://support.microsoft.com/en-us/help/4028485/windows-10-add-an-exclusion-to-windows-security
> 
> Edit : More details on this link : https://winaero.com/blog/exclusions-windows-defender-windows-10/



Wow! It worked! It loads much faster. THANK YOU!

edit: and it makes browsing through my quick-load items almost instant! Before it was taking a while (up to a minute) for folders to open. AMAZING!


----------



## pablodelcampo (Dec 2, 2020)

First of all, sorry for bring up the topic.

As I understand, and it's written in the Blake's blog, the idea is that batch resave speeds up the loading process when the nkc cache file was written in other platform different (windows or mac) and Kontakt can't read it properly. 

But EvilDragon indicates that NKC (that remains after a boot) are only relevant to kontakt player libs so, in this case, there is no benefit to make a batch resave process for non-kontakt player libs. Am I right? But kontakt player libs get loading speed benefits with this proccess? If not, Blake's statement is wrong. @EvilDragon could you bring us some light about this? Thanks a lot

The relative path is always the same, no matter the platform, no matter the folder you put the library, if you don't modify the structure of Samples/Instrument folder of the lib it's the same, only depends on your disk read rate.


----------



## EvilDragon (Dec 2, 2020)

pablodelcampo said:


> The relative path is always the same, no matter the platform, no matter the folder you put the library, if you don't modify the structure of Samples/Instrument folder of the lib it's the same, only depends on your disk read rate.



But there's a difference in forwards vs backward slashes between Win and Mac


----------



## pablodelcampo (Dec 2, 2020)

That's right. Thanks for comment. 

What I am trying to figure out is if the batch-resave process can speeds up permantly the loading process of a library or not. I read that it does several times, in soundiron page (marked as pro tip 1), in one video of Christopher Siu, etc. 

For now, the explication of Blake is logical to me so it can speed up the loading if the nkc file was generated by a different operating system than yours and Kontakt can't use it properly. (due to Win manage the relative path with "\" and the Mac with "/").

But in this case, instead of batch resave it could be faster to simply delete the nkc file because Kontakt will generate a new one for your OS using the information of nki's associated and nkx/nkr files or even Kontakt re-generate the nkc file if can't use it (if is a nkc generated by another operating system).

So, summarising, it seems there is no improvement doing batch-resave and Blake's blog tip, soundiron tip and the others are wrong. 

Please correct me if I am wrong. Thanks!


----------



## EvilDragon (Dec 2, 2020)

In the specific case that was mentioned on Blake's blog (you get a dialog with "converting samples" staying there), it's definitely gonna speed things up. It will also speed up loading of pre-K4.2 libraries (that was the reason the batch resave was implemented in the first place).


----------



## ksbkrzi (Mar 1, 2021)

Here comes the new issue: I'm trying to batch re-save, dislocated samples get gethered together, that part seems to be done ok, but when trying to save nki files, Kontakt pop-ups an error warning for each single nki file saying it can't save nki files. How come?


----------



## EvilDragon (Mar 2, 2021)

Maybe there's a version mismatch between the NKIs and Kontakt version you're running (i.e. NKIs are saved in a newer version of Kontakt from the one you're trying to batch resave from)?


----------



## ksbkrzi (Mar 2, 2021)

Yes! I tried Kontakt 4 and that was it. Thanks!


----------



## ksbkrzi (Mar 2, 2021)

I have another question: NCW or WAV? What I know is that NCW saves HD space and RAM but what about the quality of sound? And maybe some other differencies?


----------



## EvilDragon (Mar 2, 2021)

NCW is lossless audio compression. Go for it.


----------



## ksbkrzi (Mar 4, 2021)

Thanks!


----------

