Dirt cheap remote encrypted backup?

Tom · May 12, 2014, 07:51:40 AM

Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

I have my fancy little backup wrapper set to use duplicity to upload to a friends server, and S3, so now I have two local copies of my important files (not including the original location of the files), one on a raid1 in boris, another on a raid5 in boris, and the two duplicity uploads that are fully encrypted with a 4096 bit RSA key.

I am paranoid about losing my important files. I also have two copies of my media files. raid5 in mrbig, and raid5 in boris. Not quite so paranoid about my media, but I really hate to lose that stuff. It gets annoying.

My important backups are about 11GB uncompressed, and 5GB after duplicity is done with them, so it saves a lot of time uploading. each initial upload took almost three hours, and it would have been double that if I were to not compress

Lazybones · May 12, 2014, 09:05:19 AM

Just make sure with encrypted backups that you physically or independently have a method of recovering them in a total disaster where non of your primary systems are available.

Tom · May 12, 2014, 09:06:40 AM

Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt....

No yeah, I should put the key some place safe.

Lazybones · May 12, 2014, 09:08:52 AM

Quote from: Tom on May 12, 2014, 09:06:40 AM
Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt....

No yeah, I should put the key some place safe.

Place(s) Burn it to a CD and put it on a really small USB stick.. Toss those in a safety disposit box or off site from the systems you backup at least. That and make sure you have backups of the tools you are using remote... Data is no good if you can't load it.

Tom · May 12, 2014, 09:20:00 AM

Quote from: Lazybones on May 12, 2014, 09:08:52 AM
Quote from: Tom on May 12, 2014, 09:06:40 AM
Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt....

No yeah, I should put the key some place safe.

Place(s) Burn it to a CD and put it on a really small USB stick.. Toss those in a safety disposit box or off site from the systems you backup at least. That and make sure you have backups of the tools you are using remote... Data is no good if you can't load it.

yeah, I have also yet to actually try a test restore.

Lazybones · May 12, 2014, 09:34:48 AM

Quote from: Tom on May 12, 2014, 09:20:00 AM
yeah, I have also yet to actually try a test restore.

You don't have backups unless you regularly test the restore process.

Mr. Analog · May 12, 2014, 09:35:49 AM

Testing a backup restore is VERY important

I can think of at least one large organization I've worked with that never tested their backup images...

Tom · May 12, 2014, 09:40:49 AM

Quote from: Lazybones on May 12, 2014, 09:34:48 AM
Quote from: Tom on May 12, 2014, 09:20:00 AM
yeah, I have also yet to actually try a test restore.
You don't have backups unless you regularly test the restore process.

Exactly. I intend to test it today. I spent enough time on it yesterday before getting really tired... Also spent a bunch of futile effort trying to get ovirt installed. my god all of the docs are outdated and plain wrong. I finally got it compiled today. such a pain in the rear. its default settings also almost made linux oomkill some things. ran out of 16GB of memory AND 4 GB swap. sheesh. had to hand edit their build config to change it to only use one thread for one of the components. I may have an ovirt instance up today.

Quote from: Mr. Analog on May 12, 2014, 09:35:49 AM
Testing a backup restore is VERY important

I can think of at least one large organization I've worked with that never tested their backup images...

Hehe, yeah.

Darren Dirt · May 12, 2014, 09:44:19 AM

Quote from: Tom on May 12, 2014, 07:51:40 AM
Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

Not so sure the marketing/branding team was sober that day...

Tom · May 12, 2014, 09:52:19 AM

Quote from: Darren Dirt on May 12, 2014, 09:44:19 AM
Quote from: Tom on May 12, 2014, 07:51:40 AM
Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

Not so sure the marketing/branding team was sober that day...

lol!

Darren Dirt · May 12, 2014, 10:18:09 AM

imo you are far trusting of Amazon's duplicity.

Tom · May 12, 2014, 10:20:49 AM

Quote from: Darren Dirt on May 12, 2014, 10:18:09 AM
imo you are far trusting of Amazon's duplicity.

amazon doesn't make duplicity. its by the guy that made rsync. I just use its s3 backend. and its compressed and encrypted so amazon couldnt look at it easily.

Mr. Analog · May 12, 2014, 10:30:41 AM

The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon

Tom · May 12, 2014, 10:57:32 AM

Quote from: Mr. Analog on May 12, 2014, 10:30:41 AM
The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon

Yeah, and theres a reason I have more than one remote destination. just in case, you know?

Mr. Analog · May 12, 2014, 11:03:50 AM

Quote from: Tom on May 12, 2014, 10:57:32 AM
Quote from: Mr. Analog on May 12, 2014, 10:30:41 AM
The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon
Yeah, and theres a reason I have more than one remote destination. just in case, you know?

That's the way to go yeah!