My new backup and archive routine

Discussion in 'Your DAW (Digital Audio Workstation)' started by JohnG, Jan 13, 2018.

  1. whiskers

    whiskers Here for the mewsic

    352
    253
    Sep 5, 2018
    US
    That sounds much more thorough, I'd be comfortable with that. I was just pointing out a general pet peeve of mine, not digging on your solution :) Just dont forget to verify your backups occasionally and you're good!
     
    MartinH., dzilizzi and JohnG like this.
  2. Soundhound

    Soundhound Senior Member

    3,376
    832
    Feb 13, 2011
    @Dewdman - I uploaded about 8tb of sample libraries in less than a week (I think) with Backblaze. It was very quick, not sure of the exact amount of time/days. That was at our place in LA which has 150mbps upload. In Georgia where we care currently it's about 10 upload, so takes waay longer.

    When Crashplan changed to business orientation I looked around for another option and I think it was on ViC that I heard about Backblaze. I'd always found Crashplan a little difficult to manage and understand, but I'm not very tech savvy. Backblaze I find a lot more user friendly.

    And the point about backing up locally as well as in the cloud - when my Drobo crashed recently I didn't have my Time Machine going, as we'd moved recently, and I did have to recover about 300gigs of working data from Backblaze. I think it may have taken overnight to prepare and download the stuff, not bad and saved my butt.

    Next up is updating my local backup of the sample libraries. This backup stuff takes a lot of time. I'm turning into the disgruntled IT guy. :)
     
    JohnG likes this.
  3. dzilizzi

    dzilizzi I know nothing

    744
    452
    Feb 27, 2017
    Somewhere near Los Angeles
    I'm calling this my backup project. I'm hoping it will be set up by the end of December. It's involving a lot of culling of files and organizing. I tend to DL everything to my desktop, prior to copying it to my studio computer or my laptop. In someways it had been a back up. I wanted to make an image of my desktop only to realize there is about 800 GB of files in the DL folder.

    I probably need to combine it with the move everything on CD and DVD to hard drives project. It is way too much work.
     
  4. MartinH.

    MartinH. Senior Member

    313
    236
    Jun 16, 2018
    How does one best do that? I use beyond compare, should I occasionally do a full CRC checksum compare between main drive and backup drive instead of the quick compare that gets made during regular backup? I'd have to schedule something like that down to smaller chunks because it would take too long for the whole drive at once.
     
  5. whiskers

    whiskers Here for the mewsic

    352
    253
    Sep 5, 2018
    US
    from an IT perspective, I would be tempted to say attempt a restore, occasionally, but for most people that's not feasible.

    Checksums/Hashes would definitely be a good idea, in theory, but really only useful if you haven't changed your source you're comparing to. As soon as you change an item in the source of whatever you backed up, the hash is going to look different. But if you hashed right after, that would be a good integrity check and provide some peace of mind.

    Correct me if I'm wrong, BC will tell you if any files are missing/any deltas between two drives or images correct? But i don't think it would account for corrupted files? Been awhile since I've been directly involved in the sysadmin side of things, so others may have better ideas. Cheers!
     
  6. MartinH.

    MartinH. Senior Member

    313
    236
    Jun 16, 2018
    BC works on "file level", no images. That's why I like it so much. I have a huge dataset (roughly 3 TB) and make changes to multiple gigabytes of files daily and I don't need any automated versioning, because I'm used to saving versions of my files manually. All I need that backup to do is make sure all files are regularly mirrored to the backup drive (ideally daily) and I disabled deletion of "orphaned" files, so that it doesn't automatically remove a file from the backup when I accidentally delete it on the main drive. A full restore would be entirely impractical and impossible to manually check if the files are all still fine. I was thinking as long as I do an occasional crc check with the file comparison (which so far I have never done addmittadly), it should protect against "data rot" on the backup drive. Is that not the case?

    I do a fairly regular "drive rotation" though, to prevent total drive failure in the first place. I buy a new drive every ~2 years, use it as primary backup drive for a few months, then switch the roles of active and backup drive with the main drive. That way my main drive is always less than 2 years old and always was "battletested" for 2-3 months before it got used for "critical" work. I've based those numbers on the statistical clustering of hard drive deaths along the time axis, as demonstrated both by the big google study on hard drive longevity (released many years ago) and my own personal experience with drives.
     
  7. whiskers

    whiskers Here for the mewsic

    352
    253
    Sep 5, 2018
    US
    Sounds right. DR (disaster recovery) is not really my area of expertise, but my understanding is periodically running the drives and checking the contents will help bit rot/data decay/whatever you want to call it. When running the drives they typically perform SMART checks and can mark bad sectors, etc. from being used (think chkdisk or fsck), so it's definitely better to occasionally check the contents, for sure.
     
    MartinH. likes this.
  8. LinusW

    LinusW Senior Member

    213
    130
    Oct 23, 2017
    Kalmar, Sweden
    I have unlimited space at Jottacloud. Everything gets backed up there. As cheap as Backblaze, less regulations.
     
  9. tack

    tack Damned Dirty Ape

    2,211
    1,751
    Aug 3, 2013
    For services offering cheap unlimited storage, it's very likely the case that your data is accessible to the storage provider. This is how they can offer unlimited storage: by relying on the fact that a good chunk of data can be deduplicated. (If everyone here uploads their copy of Monster Library X, they only need to actually store 1 copy of it.) This is invariably coupled by the requirement to use their proprietary software, rather than a third party backup program (because third party software will be able to properly encrypt the data).

    I'd be quite interested to learn about exceptions to this: are there any services offering unlimited storage for around say $10USD a month that present, for example, and S3 API interface for use of third party backup software, and have been offering unlimited for more than 3 years? (The criteria of offering it for more than 3 years is important, because many unlimited storage offerings have come and gone over the years, but they tend to vanish when businesses realize the product isn't sustainable.) These are the really interesting options IMO, especially ones that perform well.

    If data privacy isn't a requirement for you (and/or perhaps you have a higher priority conflicting requirement that you want to be able to access your data via web browser), then something like Backblaze Personal Backup (not B2) is hard to beat.
     
  10. Stevie

    Stevie Senior Member

    1,325
    72
    Oct 20, 2007
  11. Synetos

    Synetos Gear Addict

    63
    20
    Feb 1, 2016
    An option is to have your own encryption software. Encrypt the data you want to store on the cloud...before you push it up to your cloud service provider. Then, no one will be looking at it. Added steps, but for things I for sure want private, I encrypt myself using Boxcryptor.

    I chose Sync.com for my cloud provider because they allow me to encrypt and hold the keys, so they are not scanning my data. But, for things like taxes or other financial stuff I want secure, I just encrypt it myself anyway.
     
    whiskers likes this.
  12. tack

    tack Damned Dirty Ape

    2,211
    1,751
    Aug 3, 2013
    Well, you're using their software, so you think they're not holding the keys and can't scan your data. And you have to trust that they're implementing their crypto properly, because it's proprietary and closed source and can't be audited by unbiased third parties without an expensive (and illegal in some jurisdictions) effort of reverse engineering.

    If you're going through the trouble of creating encrypted local copies of all your data (and continuously reencrypting when there are changes) then, indeed, you can just point your cloud backup software at that.

    Alternatively, if data privacy isn't important to one's backup strategy (which is a personal choice), that opens up plenty of options to choose from.
     
    MartinH. and whiskers like this.
  13. whiskers

    whiskers Here for the mewsic

    352
    253
    Sep 5, 2018
    US
    You, I like you.
     
  14. Anders Wall

    Anders Wall 55°28'54"N 13°30'44"E

    448
    232
    Sep 26, 2013
    Sweden
    Throwing this in the mix.
    https://www.pcloud.com/cloud-storage-pricing-plans.html
    For €350 you get a 2Tb cloud-drive, you only pay once.
    They also provide some solid encryption tools.
    (I'm no IT-guru/tech so don't take my word for it, but it sure looks solid)

    Cheers,
    Anders
     
  15. Synetos

    Synetos Gear Addict

    63
    20
    Feb 1, 2016
    A single pay for 2TB that costs about the same as an external drive sounds interesting to me. Thanks for throwing it out there. How secure your data is, will always be a question on cloud sites, as Tack pointed out earlier. I still like the idea of a secondary encryption for really sensitive data.
     
    Anders Wall likes this.
  16. GtrString

    GtrString Senior Member

    253
    148
    Dec 10, 2016
    Denmark
    This backup storage thing could end up being as expensive as hardware outboard gear.. wait.. it IS hardware outboard gear!
     

Share This Page