# Archiving - whats the best method?



## jonasr (Jan 7, 2018)

Hi, wondering whats the take on archiving here, do you use Raid? I don't really know what to search for, tried both archiving and Raid but not much come up. 

Or double sets of external drives? 2.5mm? 3.5mm?

What do you do with your kontakt instruments, saving them both as the end product (I guess thats is your active sample hard drive that your stream into your DAW) or as the rar archives you downloaded as you purchased them?

Right now I only save mine as kontakt instruments (the library kontakt folder) on two drives (mirrored) that I use to my DAW for either of my two machines.


----------



## storyteller (Jan 7, 2018)

I'm sure most people's responses will vary a bit, but as a general rule of thumb, you want to have a backup that exists offsite as well as one in your studio and minimize the potential for data failure by removing RAID when possible (or using something like RAID 5). If having back-ups both onsite and offsite can't happen logistically, then two types of backups is an okay alternative.

Speaking of my personal backup solution, I have a minimum of 14 external drives (including backup drives) connected at any given time on my main DAW. 5 of these are SSDs that contain sample libraries. One drive is an external SSD audio drive for sessions/projects. These drives (plus the system hard drive are considered my daily operating requirements on my main DAW). So that's the world view before backups take place...

My backup strategy follows this logic:

I have *two separate Time Machine backup disks* that run in the early A.M. hours. I'm on OSX, but on Windows, I would still follow the same logic by backing up the system drive nightly.
I have *one primary RAID 5 archive drive (4 drives)* connected via thunderbolt. Raid 5 allows for a disk failure and replacement/recovery to occur without data loss. *This drive contains all of my archived audio sessions as well as duplicate copies of everything I have on my 5 sample library SSDs.* It also serves as an archive of all installers, pictures, videos, documents, and miscellaneous projects I have completed as well as the primary store for current video projects I may be working on.
*The audio/session external drive is backed up nightly to a temporary archive folder on the RAID 5 drive.* This is backup #1 of my primary audio/session drive.
The RAID 5 drive is large and exceeds the size of any single disk drive available. I originally set out to back it nightly up via a versioned backup to a 2-disk RAID 0 stripe. But, after numerous hardware enclosure failures (note: _not_ disk failures) and having to rebuild backups from scratch, I've since abandoned this idea and now* I* *backup the RAID 5 by dividing up the data across multiple 6TB spindle drives nightly after the audio drive backs up to the RAID 5.* I also make this a "versioned" backup as well. It is slower this way (compared to RAID striping), and I may return to raid backups in the future to increase speed, but for now this is working out okay. Word of advice... Western Digital MyBook enclosures are not reliable for Raid 0. I've been through 3 so far.
When this storage space is maxed out, portions will be offloaded to separate backup disks and removed from the primary RAID 5 archive drive.
Overall, this gives me 2 backups of Time Machine system images, 2 backups of my audio drive, 2 backups of my sample drives, and 2 copies of all projects that all can be divided up onsite/offsite as required. I used to do online Backblaze backups, but if you suppress the upload time period to a few hours at night, the nightly backups rarely get completed due to the data size (so I stopped that for now).

Hope this helps! I'm sure others will share some other great backup strategies as well! My backup strategy is a very fluid and shifting process as I discover better ways and more efficient ways to do things (especially when increasing redundancy and reducing data-loss risk).


----------



## Gerhard Westphalen (Jan 9, 2018)

For archiving I put things on a NAS where it automatically gets backed up to another network drive once a week. Then I also have an external drive that stays disconnected which I back up to every couple of months. I'd like to raid 1 my main NAS at some point to have some redundancy. Everything eventually ends up on 3 separate locations but it is all in the house. 

Outside of my main rig, I only have samples on the single NAS. That can mostly be redownloaded if needed.

Is anyone using an online archiving service like Amazon Glacier? Something where you can get several TB without it costing a fortune?


----------



## jonasr (Jan 9, 2018)

Thank you so much! This is really terrific information. Seems like you have a very complex system that makes you "bulletproof"(lack of a better word) to data loss. 


As an early step you think a RAId 1 system can provide a stepping stone to building up towards a better backup system?




storyteller said:


> I'm sure most people's responses will vary a bit, but as a general rule of thumb, you want to have a backup that exists offsite as well as one in your studio and minimize the potential for data failure by removing RAID when possible (or using something like RAID 5). If having back-ups both onsite and offsite can't happen logistically, then two types of backups is an okay alternative.
> 
> Speaking of my personal backup solution, I have a minimum of 14 external drives (including backup drives) connected at any given time on my main DAW. 5 of these are SSDs that contain sample libraries. One drive is an external SSD audio drive for sessions/projects. These drives (plus the system hard drive are considered my daily operating requirements on my main DAW). So that's the world view before backups take place...
> 
> ...


----------



## jonasr (Jan 9, 2018)

Hi, thank you for your input. Truly appreciate it?

I'm thinking of RaID 1 to a NAS, is synology a good brand?

You mean redownloaded from San online NAS or from the producer of the smallest website?

Is Amazon glacier available for any country? I'm in Sweden.




Gerhard Westphalen said:


> For archiving I put things on a NAS where it automatically gets backed up to another network drive once a week. Then I also have an external drive that stays disconnected which I back up to every couple of months. I'd like to raid 1 my main NAS at some point to have some redundancy. Everything eventually ends up on 3 separate locations but it is all in the house.
> 
> Outside of my main rig, I only have samples on the single NAS. That can mostly be redownloaded if needed.
> 
> Is anyone using an online archiving service like Amazon Glacier? Something where you can get several TB without it costing a fortune?


----------



## Gerhard Westphalen (Jan 9, 2018)

jonasr said:


> I'm thinking of RaID 1 to a NAS, is synology a good brand?


They're a great brand. Keep in mind that raid 1 is about redundancy, not backup. 



jonasr said:


> You mean redownloaded from San online NAS or from the producer of the smallest website?


Redownload from the sample companies.


----------



## storyteller (Jan 10, 2018)

jonasr said:


> Thank you so much! This is really terrific information. Seems like you have a very complex system that makes you "bulletproof"(lack of a better word) to data loss.
> 
> 
> As an early step you think a RAId 1 system can provide a stepping stone to building up towards a better backup system?


To answer your question and also add a little clarity to what @Gerhard Westphalen was saying about Raid 1... it is a good method for redundancy, but should not be viewed as a “second backup.” The reason is that the backup likely still exists in a single enclosure and would be subject to a single point of failure. But - it does give you extra redundancy if one drive fails! (Usually at least). Sometimes a failure will zap both drives from a data standpoint even if only one drive fails from a hardware failure.

That said, it is a good method for beginning your first backup plan. If I were only buying two drives and getting started with a backup strategy today, I would consider Raid 1 but I would likely go with a single drive backup with a second drive serving as a backup of the primary backup. But if you decide to go with Raid 1, A lot of people rave about synology... In either of these strategies you will still be limited to the transfer speed of a single drive. In the case of a NAS, you will also be subject to the speed of your home network. For big backups, this can get painfully slow if your nAs strategy is not setup correctly in the beginning stages.


----------



## Gerhard Westphalen (Jan 10, 2018)

storyteller said:


> The reason is that the backup likely still exists in a single enclosure and would be subject to a single point of failure. But - it does give you extra redundancy if one drive fails! (Usually at least). Sometimes a failure will zap both drives from a data standpoint even if only one drive fails from a hardware failure.


That's not entirely the reason it shouldn't be considered as a backup strategy. It's because the data is identical on both drives. Ideally in a backup, there should be some sort of versioning or delay buffer. If something happens and you want to revert to how something was yesterday, then raid 1 wouldn't help you since it's all the same there. There's no older backup version. If someone (accidentally or intentionally) deletes something from the drive, it's gone on both. No "oops let's undo that." If you had say, a second backup that only runs once a day then it wouldn't be affected until the end of the day and you could still get the data back.


----------



## storyteller (Jan 10, 2018)

Gerhard Westphalen said:


> That's not entirely the reason it shouldn't be considered as a backup strategy. It's because the data is identical on both drives. Ideally in a backup, there should be some sort of versioning or delay buffer. If something happens and you want to revert to how something was yesterday, then raid 1 wouldn't help you since it's all the same there. There's no older backup version. If someone (accidentally or intentionally) deletes something from the drive, it's gone on both. No "oops let's undo that." If you had say, a second backup that only runs once a day then it wouldn't be affected until the end of the day and you could still get the data back.


Oh - that's true. Everything I do is on versioning and delays, so I assumed that even Raid 1 in the example here was versioned. But for me, the biggest question is that it contains a single point for potential failure assuming everything is running versioned and nightly, etc. That's more of the angle from which I was approaching my comments. I apologize for miscommunication there.


----------



## wickedw (Jan 11, 2018)

I'm running a synology box with 2 disks in raid 1. This is by no means a very good solution but it's an affordable one. You can go to better models of synology ( or something similar ) and have more disks which would increase your raid options. Raid 5 would be the best option.

I have written fairly simple scripts that start the backups for versioning and creating archives of old backups. (powershell scripts since I'm on windows, but would be doable in any language with a bit of know how. or google.) 

For offsite I could think about using something in Amazon Web Services but you need to know your away around it and secure it properly. It could become a fairly expensive monhtly bill aswell when you have a lot of data - but it might be worth it. If you have another physical location yourself however then do it there though.


----------



## kilgurt (Jan 11, 2018)

Excellent advice from the pros in this free tutorial: https://www.puremix.net/video/session-backup-strategies-and-workflows.html


----------



## jonasr (Jan 16, 2018)

kilgurt said:


> Excellent advice from the pros in this free tutorial: https://www.puremix.net/video/session-backup-strategies-and-workflows.html



Thank you!


----------

