# One SSD to load them all - Good idea?



## MoeWalsaad (Dec 7, 2018)

Hello,
I want to improve my Sample library loading times and performance, I brought one new large SSD,
So I wonder whether it's a good idea to run all my main VST libraries on it, or is it better to split between several drives (includes HDDs)?

I know that loading too many instruments from one HDD is not a good idea, so how about SSD's reading mechanism, will it handle the load?

Thanks in advance!


----------



## chocobitz825 (Dec 7, 2018)

MoeWalsaad said:


> Hello,
> I want to improve my Sample library loading times and performance, I brought one new large SSD,
> So I wonder whether it's a good idea to run all my main VST libraries on it, or is it better to split between several drives (includes HDDs)?
> 
> ...



I can only speak to my experience that a RAID SSD drive over thunderbolt 3 has generally improved my read speeds compared to HDD.


----------



## JohnG (Dec 7, 2018)

If you own it, go ahead and use it. I usually buy multiple, smaller drives, but it doesn't seem to make all that much difference with SSDs.

The only reason to RAID an SSD is to create a larger volume so you don't have to split large libraries. Otherwise when you update it can be a pain.


----------



## MartinH. (Dec 7, 2018)

MoeWalsaad said:


> I want to improve my Sample library loading times and performance,



I would like to do that as well, my template takes like 5 minutes or so to load. I'm 99% sure that the number of SSD drives that you have doesn't matter as long as you have SSDs instead of HDDs. What else is there to try in Kontakt settings? Or Reaper's settings?


----------



## ceemusic (Dec 7, 2018)

Also I'd go with two 500GB SSDs than a single 1TB SSD.
2 drives sustaining ~540MB/Sec instead of a single sustaining ~540MB/Sec.


----------



## Damarus (Dec 7, 2018)

Actually, a larger SSD is better than multiple. I don't see any reason why you would split it up over multiple drives. Larger SSD's have more memory onboard, making them slightly faster and more efficient. 

Again, unless you're setting up a RAID config I don't see any reason to split up drives for the same purpose. There's no speed benefit.


----------



## ceemusic (Dec 7, 2018)

2 because for streaming samples it yields better performance.


----------



## Damarus (Dec 7, 2018)

ceemusic said:


> 2 because for streaming samples it yields better performance.



Why would 2 separate drives yield better performance than 1?


----------



## ceemusic (Dec 7, 2018)

read my post above


----------



## Damarus (Dec 7, 2018)

Yeah, I read it but it doesn't necessarily work like that... One drive would be saturated over the other drive.

Price per GB is cheaper on Larger SSD's. Larger SSD's are faster and have a longer lifespan. Fewer parts, easier organization. SSD's start to slow down the more they fill up.


----------



## JohnG (Dec 7, 2018)

Damarus said:


> SSD's start to slow down the more they fill up.



That is wrong. You can fill SSDs almost to 100% and they don't slow down to any meaningful extent.

If you want faster load times, consider PCIe-based storage. That can make a difference.


----------



## chocobitz825 (Dec 7, 2018)

this conversation never seems to resolve well as most of it comes down to differences in opinion about what is significant enough gain to warrant the cost and risk of ssds. Obviously PCIe based storage is ideal if speed is a concern, but the cost is not great. one thunderbolt 3 ssd drive is probably enough for most people, but as an anecdotal reference, I and many others I know who have designated RAID library drives have found the speed increase significant enough to put it above HDD and single SSDs. There are a number of factors and real world results may vary, but my read speeds increased enough for SSD, and raid to be justified.


----------



## JohnG (Dec 7, 2018)

I have never seen any data that supports the claim that RAID makes a meaningful difference to music composers / those using samples. This is true for performance or load times for SSDs or for HDDs, now that we're talking about it.

Do you have data that does support that? Not just opinion / guess / hypothesis?

Using RAID does raise the likelihood of a failure, so for that reason, I don't do it. 

Certainly, the PCIe drives are very expensive, but who cares? They last a long time and they actually DO make a difference in performance that is noticeable, even with very demanding string libraries.

The cost / performance tradeoff is subjective. Whether or not there is a meaningful improvement -- greater than, say, 10% -- is less so.


----------



## jbuhler (Dec 7, 2018)

So how many simultaneous samples can one effectively run through a 5gbps USB3 port? For most of us, that would seem to be the most relevant bottleneck for SSDs and whether you need to divide your samples across SSDs (and direct USB3 ports).


----------



## chocobitz825 (Dec 7, 2018)

I'll just provide the results of quick test on a few drives I have. (all are in Akitio Thunderbolt enclosures)


Thunderbolt 2 SSD RAID: 636 MB/s Write 745MB/s Read (2 drives)
Thunderbolt 3 SSD RAID: 416 write 1294 Read (4 drives)
Thunderbolt 3 SSD RAID (with cheaper drives): 416 write 1013 read (4 Drives and thunderbolt 3 enclosures are daisy chained)


i don't have any more HDDs on hand to test, but in general I get sufficient read speeds from the drives. I have backups of all the libraries so loss form the RAID is not a concern, and since they're only designated for reading libraries, and not recording or anything, Ive not had a drive fail on me over the last 3~4 years. Again, just anecdotal, but in my case, they are faster than the HDD alternative, and perform faster for me on my mac than single SSDs did. no doubt i would get much more from PCIe, I just haven't seen the need. The performance difference is of course significant, but with how SSD costs went down, this was a sufficient transition for the time being.

*EDIT: just a reference Internal iMac Pro Speed: 2931 Write 2487 Read*


----------



## Damarus (Dec 7, 2018)

chocobitz825 said:


> I'll just provide the results of quick test on a few drives I have. (all are in Akitio Thunderbolt enclosures)
> 
> 
> Thunderbolt 2 SSD RAID: 636 MB/s Write 745MB/s Read (2 drives)
> ...



Fair enough, if you have the enclosure and the drives laying around.

for OP, just get one large m.2 PCIe SSD. They are cheap right now


----------



## chocobitz825 (Dec 7, 2018)

Damarus said:


> Fair enough, if you have the enclosure and the drives laying around.
> 
> for OP, just get one large m.2 PCIe SSD. They are cheap right now



I am curious, what are some of the PCie prices and solutions for people who need to use them as external storage? would you connect them via USB 3.0/3.1 or thunderbolt?


----------



## MartinH. (Dec 8, 2018)

MartinH. said:


> What else is there to try in Kontakt settings? Or Reaper's settings?


There's an option to do the sample loading in the background, I'll try that. I don't need everything to play fine as soon as I open my template, I'd be happy if I could start doing midi editing sooner and would be fine with waiting a couple minutes till all the samples are loaded for a full playback. 




chocobitz825 said:


> I'll just provide the results of quick test on a few drives I have. (all are in Akitio Thunderbolt enclosures)
> 
> 
> Thunderbolt 2 SSD RAID: 636 MB/s Write 745MB/s Read (2 drives)
> ...



Did you benchmark sequential read performance of one large data-chunk? Because that data would be meaningless for the sample library usecase. It's been too many years since I touched a proper disk benchmark tool, but iirc it's called something like "queue depth" which tests parallel reads of multiple files. On my old *HDD *the sequential read was around 100MB/s and the value for a lot of small reads in random locations (e.g. like playing back many different sample library samples) went down to the single digit range. Doing the same benchmark on an SSD you see barely any difference between the two values, because SSDs don't have significant seek time for drive head repositioning, because they have no mechanical drive heads...

On HDD drives I would actually expect sample library streaming performance to go DOWN from a raid 0 setup, because it should effectively double the amount of times the drive head needs to reposition because every sample is distributed over 2 drives. You might see a minor performance increase over 1 single HDD drive, but I would estimate 2 non-raid drives to perform better if you split your libraries accross both drives. And for SSDs my prediction is that it doesn't matter because you won't be bottlenecked by the drives under normal circumstances. 

One second of 16 bit 48khz stereo audio uncompressed should be a little less than 200kb, on any regular SSD you should easily get 300 MB/s, that's roughly 1500 parallel sample voices being played. Probably more in reality because they use lossless compression, shifting i/o bandwidth load to CPU decompression load. Sure, with multiple mic positions and lots of backed in reverb you might get there, but I really don't think you need a raid 0 setup for SSDs. I'll happily stand corrected if you can provide some hard data that proves me wrong.

Side note: if anyone is using SSDs in external enclosures I think it's worth doing a disk benchmark that tests the parallel random reads because with USB 2 iirc there was a considerable bandwidth loss for certain types of operations that came from the "overhead" of shoving the instructions over USB 2. Wouldn't be entirely surprised if there's some performance degradation on USB 3, even if the theretical maximum bandwidth couldn't get saturated by the drives max read/write bandwidth. Just a hunch though, might be fine after all. I only use internal SATA connectors.


----------



## chocobitz825 (Dec 8, 2018)

MartinH. said:


> There's an option to do the sample loading in the background, I'll try that. I don't need everything to play fine as soon as I open my template, I'd be happy if I could start doing midi editing sooner and would be fine with waiting a couple minutes till all the samples are loaded for a full playback.
> 
> 
> 
> ...



Sure thing. pulled out the various library, backup, archive and data drives I could find.

Samsung T5 1TB via USB-C: 
*Seq* _Read _40MB/s _Write _21.11MB/s 
*4KQD32 *_Read_ 11.25 MB/s _Write_ 9.37 MB/s 

WDC HDD RAID 1 via Thunderbolt 2:
*Seq* _Read 1_40MB/s _Write_ 97.75MB/s 
*4KQD32 *_Read_ 0.811 MB/s _Write_ 1.862 MB/s 

2TB SSD 2 Drive RAID via Thunderbolt 2: 
*Seq *_Read_ 682 MB/s Write 446MB/s 
*4KQD32* _Read_ 92.74 MB/s _Write_ 30 MB/s 

8TB SSD (4 Drives) RAID #1 via Thunderbolt 3:
*Seq* _Read _1443 MB/s _Write _343MB/s 
*4KQD32 *_Read_ 96 MB/s _Write_ 76 MB/s 

8TB SSD RAID #2 via Thunderbolt 3:
*Seq* _Read _1470 MB/s _Write _355 MB/s 
*4KQD32 *_Read_ 94.13 MB/s _Write_ 68 MB/s 

Apple 1TB PCie SSD (2 drive RAID by default if i recall)
*Seq* _Read _2725 MB/s _Write _2765 MB/s 
*4KQD32 *_Read_ 237 MB/s _Write_ 160 MB/s


----------



## MartinH. (Dec 8, 2018)

chocobitz825 said:


> Sure thing. pulled out the various library, backup, archive and data drives I could find.
> 
> Samsung T5 1TB via USB-C:
> *Seq* _Read _40MB/s _Write _21.11MB/s
> ...




Awesome! Thanks a lot for testing, that's some data that we can use! That 4KQD32 line is the one I'd focus on, because it should be closer to how a drive behaves when streaming lots of samples from it, than durint the sequential read benchmark. Sequential read/write would be interesting for backups or first time install speed of libraries.


The first drive is only getting USB 2.0 speed, might want to look into that if it's supposed to be 3.0. Some mainboards have both USB 2 and 3 connectors on board (mine does). Might be plugged into the wrong port maybe? It's also a good example of what I said about the USB 2 overhead having big impact on bandwidth, because there's no way you're bottlenecked by the SSD itself down to 10 MB/s.

HDD raid gives unusable performance (less than 1MB/s read), just as I expected. I wouldn't be surprised if the Raid is actually detrimental here. On my smallest HDD I get 1.79 MB/s read an 1.97 MB/s write.

All the thunderbolt raids give usable speeds that should cause you no problems, even though they are roundabout *10 times slower* than sequential read/write. I don't know anything about thunderbolt, so I can't say what's causing this. I would have expected higher speeds on regular internal SATA connected non-raid SSD drives.


Here are speeds for my SSD drives:

Samsung 850 Evo 500 GB:
*4KQD32 *_Read _245 MB/s _Write _214 MB/s

M4-CT128M4SSD2 128 GB:
*4KQD32 *_Read 190 _MB/s _Write 149 _MB/s

I'm not sure both are connected to SATA II ports, the Samsung one is, but the other one might not be. But it's a very old drive, I doubt it would make a big difference.

Personally the conclusion that I would draw from all these benchmarks is, that everything other than a normal internal SSD connected via SATA is either detrimental or doesn't see the real-world performance increase that a sequential read/write benchmark (like they use in all the advertisement material) would promise.


P.S.: In my old rig I used to run a "SATA port multiplier" that split one SATA port to several SATA ports that shared the bandwidth of one. I haven't followed the developments of those, but if someone is keen on getting more SSDs than available SATA connectors into one case, it might be worth looking into.


----------



## EvilDragon (Dec 8, 2018)

MartinH. said:


> That 4KQD32 line is the one I'd focus on, because it should be closer to how a drive behaves when streaming lots of samples from it



Actually, no. At least @tack found out in his testing that Kontakt is not really using more than QD1, only here and there it goes to QD2...


----------



## jbuhler (Dec 8, 2018)

MartinH. said:


> internal SSD connected via SATA is either detrimental or doesn't see the real-world performance increase that a sequential read/write benchmark


What’s the real world rule of thumb to draw from this for connecting SSDs? When are you likely to encounter problems streaming samples from disk over USB3? That is, does it make sense to divide libraries across several USB3 externals or have the samples on a minimal number of disks?


----------



## chocobitz825 (Dec 8, 2018)

MartinH. said:


> Awesome! Thanks a lot for testing, that's some data that we can use! That 4KQD32 line is the one I'd focus on, because it should be closer to how a drive behaves when streaming lots of samples from it, than durint the sequential read benchmark. Sequential read/write would be interesting for backups or first time install speed of libraries.
> 
> 
> The first drive is only getting USB 2.0 speed, might want to look into that if it's supposed to be 3.0. Some mainboards have both USB 2 and 3 connectors on board (mine does). Might be plugged into the wrong port maybe? It's also a good example of what I said about the USB 2 overhead having big impact on bandwidth, because there's no way you're bottlenecked by the SSD itself down to 10 MB/s.
> ...



One of the reasons I avoid USB-C. Seems that the ports on my displays USB-C ports are all equivalent of USB 2 as you said. Sorting out each port and if they're actually pushing out 3.0/3.1 speeds is a pain, so I've tried to stick to Thunderbolt drives when dealing with libraries and large files. I think there's no argument that PCie is the fastest way to go, but given the amount of space you can get on current SATA SSDs for the cost with sufficient speed for the job, I'm fine with sticking to SATA for the time being. I have no hiccups even when running 200+ instruments at once. Eventually, looking toward the future I might move up, but its not a necessity quite yet.


----------



## EvilDragon (Dec 8, 2018)

chocobitz825 said:


> Seems that the ports on my displays USB-C ports are all equivalent of USB 2 as you said.



That in most cases depends on the cable rather than port itself.


----------



## chocobitz825 (Dec 8, 2018)

jbuhler said:


> What’s the real world rule of thumb to draw from this for connecting SSDs? When are you likely to encounter problems streaming samples from disk over USB3? That is, does it make sense to divide libraries across several USB3 externals or have the samples on a minimal number of disks?



wouldn't that slight depend on the I/O options you have? How many ports do you have/How many are you using? what are their actual speeds? Part of the reason I opted for daisy chained thunderbolt enclosures was because the singular daisy chain-able drives cost a lot more, and Using all my ports, or USB hubs for individual drives just seemed less convenient.




EvilDragon said:


> That in most cases depends on the cable rather than port itself.


Double checked the specs and it rates the ports as USB-C (480 Mbps) ports. if recall thats 2.0 equivalent no?


----------



## jbuhler (Dec 8, 2018)

chocobitz825 said:


> wouldn't that slight depend on the I/O options you have? How many ports do you have/How many are you using? what are their actual speeds? Part of the reason I opted for daisy chained thunderbolt enclosures was because the singular daisy chain-able drives cost a lot more, and Using all my ports, or USB hubs for individual drives just seemed less convenient.


I have four USB3 directly to the computer and another four USB3 that are on a thunderbolt2 hub, but that hub also carries video for an external monitor. I also have one open thunderbolt2 port. I have five 1TB SSDs, and my working set-up has been one SSD per USB3 port except two of the SSDs are mounted as separate bays in a single USB3 enclosure. Most of the SSDs run through the thunderbolt2 hub, because some of my peripherals refuse to work from hubs. I also run my audio interface through USB. In the real world, I don't have any issues with this set-up, but I'm also running out of ports and so will need to rearrange things. I'm also wondering what the best options are when I work from my laptop, which has only two USB3 ports and one thunderbolt port.


----------



## MartinH. (Dec 8, 2018)

chocobitz825 said:


> Double checked the specs and it rates the ports as USB-C (480 Mbps) ports. if recall thats 2.0 equivalent no?


Correct. 



EvilDragon said:


> Actually, no. At least @tack found out in his testing that Kontakt is not really using more than QD1, only here and there it goes to QD2...


Thanks for pointing that out! Which of the Crystal Disk Mark benchmarks would be the closest then? 4K random QD 1? 512K random? The 4kQD1 is actually MUCH slower on my faster SSD than 4kQD32 - 37 MB/s read instead of 245 MB/s. The 512k random test is almost as high as sequential. 




jbuhler said:


> What’s the real world rule of thumb to draw from this for connecting SSDs? When are you likely to encounter problems streaming samples from disk over USB3? That is, does it make sense to divide libraries across several USB3 externals or have the samples on a minimal number of disks?



I don't have experience with USB 3 and haven't seen a benchmark posted here the uses the full USB 3 bandwidth yet. For USB 2 the bandwidth was shared per controller among all connected ports, and there was no easy way of looking at the port and knowing which controller it connects to. You might have one feeding all, you might have two feeding different ports, who knows. Also I don't know how that's handled with USB 3, I never had a reason to look into it.


----------



## MartinH. (Dec 8, 2018)

P.s.: I tried setting Kontakt to load samples in the background and that seems to have cut down project loading time to 2 minutes with playback being a bit dodgy right after opening, but not nearly as bad as I anticipated. Glad to have saved a bit of waiting time there. 

I tried lowering preload buffer size but that either didn't work or didn't save on loading time. Anything else I could try?

Hypothetically speaking... would batch-resaving sample libraries as uncompressed samples improve the project load times? I don't want to do that, but I'm curious.


----------



## chocobitz825 (Dec 8, 2018)

switched to a direct USB 3.0 port.

Samsung T5 1TB via USB-C 3.0: 
*Seq* _Read _267.9MB/s _Write _206.9MB/s 
*4KQD32 *_Read_ 18.62 MB/s _Write_ 81.68 MB/s


----------



## MartinH. (Dec 8, 2018)

chocobitz825 said:


> switched to a direct USB 3.0 port.
> 
> Samsung T5 1TB via USB-C 3.0:
> *Seq* _Read _267.9MB/s _Write _206.9MB/s
> *4KQD32 *_Read_ 18.62 MB/s _Write_ 81.68 MB/s



Very interesting! I'd explain that lower read than write performance on 4KQD32 by the bus having to transfer data both ways: send the read instruction and receive the data back, instead of just sending write instruction and data one way. Just a hunch though, I don't know much about USB 3. But this is actually far slower than I expected. Thanks for benchmarking this!


----------



## EvilDragon (Dec 8, 2018)

MartinH. said:


> Which of the Crystal Disk Mark benchmarks would be the closest then? 4K random QD 1? 512K random?



4k QD1 random reads would be closest to what Kontakt uses I think.



MartinH. said:


> P.s.: I tried setting Kontakt to load samples in the background and that seems to have cut down project loading time to 2 minutes



You lived this whole time *without *background loading active? Boy, oh boy 



MartinH. said:


> I tried lowering preload buffer size but that either didn't work or didn't save on loading time.



Lowering DFD buffer size won't really make things load faster, but, if all your libraries are on SSDs, you can freely cut down that DFD buffer size globally, to 6 or 12 KB. Saves RAM.



MartinH. said:


> Hypothetically speaking... would batch-resaving sample libraries as uncompressed samples improve the project load times?



I highly doubt it.


----------

