Dirt cheap remote encrypted backup?

Started by Tom, May 12, 2014, 07:51:40 AM

Previous topic - Next topic

Tom

Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

I have my fancy little backup wrapper set to use duplicity to upload to a friends server, and S3, so now I have two local copies of my important files (not including the original location of the files), one on a raid1 in boris, another on a raid5 in boris, and the two duplicity uploads that are fully encrypted with a 4096 bit RSA key.

I am paranoid about losing my important files. I also have two copies of my media files. raid5 in mrbig, and raid5 in boris. Not quite so paranoid about my media, but I really hate to lose that stuff. It gets annoying.

My important backups are about 11GB uncompressed, and 5GB after duplicity is done with them, so it saves a lot of time uploading. each initial upload took almost three hours, and it would have been double that if I were to not compress :(
<Zapata Prime> I smell Stanley... And he smells good!!!

Lazybones

Just make sure with encrypted backups that you physically or independently have a method of recovering them in a total disaster where non of your primary systems are available.

Tom

Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt.... ;)

No yeah, I should put the key some place safe.
<Zapata Prime> I smell Stanley... And he smells good!!!

Lazybones

Quote from: Tom on May 12, 2014, 09:06:40 AM
Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt.... ;)

No yeah, I should put the key some place safe.

Place(s) Burn it to a CD and put it on a really small USB stick.. Toss those in a safety disposit box or off site from the systems you backup at least. That and make sure you have backups of the tools you are using remote... Data is no good if you can't load it.

Tom

Quote from: Lazybones on May 12, 2014, 09:08:52 AM
Quote from: Tom on May 12, 2014, 09:06:40 AM
Yeah, turns out my key is synced in the backups so I should be able to rescue it.... waiitttt.... ;)

No yeah, I should put the key some place safe.

Place(s) Burn it to a CD and put it on a really small USB stick.. Toss those in a safety disposit box or off site from the systems you backup at least. That and make sure you have backups of the tools you are using remote... Data is no good if you can't load it.
yeah, I have also yet to actually try a test restore.
<Zapata Prime> I smell Stanley... And he smells good!!!

Lazybones

Quote from: Tom on May 12, 2014, 09:20:00 AM
yeah, I have also yet to actually try a test restore.
You don't have backups unless you regularly test the restore process.

Mr. Analog

Testing a backup restore is VERY important

I can think of at least one large organization I've worked with that never tested their backup images...
By Grabthar's Hammer

Tom

Quote from: Lazybones on May 12, 2014, 09:34:48 AM
Quote from: Tom on May 12, 2014, 09:20:00 AM
yeah, I have also yet to actually try a test restore.
You don't have backups unless you regularly test the restore process.
Exactly. I intend to test it today. I spent enough time on it yesterday before getting really tired... Also spent a bunch of futile effort trying to get ovirt installed. my god all of the docs are outdated and plain wrong. I finally got it compiled today. such a pain in the rear. its default settings also almost made linux oomkill some things. ran out of 16GB of memory AND 4 GB swap. sheesh. had to hand edit their build config to change it to only use one thread for one of the components. I may have an ovirt instance up today.


Quote from: Mr. Analog on May 12, 2014, 09:35:49 AM
Testing a backup restore is VERY important

I can think of at least one large organization I've worked with that never tested their backup images...
Hehe, yeah.
<Zapata Prime> I smell Stanley... And he smells good!!!

Darren Dirt

Quote from: Tom on May 12, 2014, 07:51:40 AM
Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

Not so sure the marketing/branding team was sober that day...
_____________________

Strive for progress. Not perfection.
_____________________

Tom

Quote from: Darren Dirt on May 12, 2014, 09:44:19 AM
Quote from: Tom on May 12, 2014, 07:51:40 AM
Look no further: duplicity and Amazon S3.

Duplicity is a neat little librsync based tool that does full and incremental backups, and supports a number of backend storage types, including ftp, ssh, and S3.

Not so sure the marketing/branding team was sober that day...

lol!
<Zapata Prime> I smell Stanley... And he smells good!!!

Darren Dirt

imo you are far trusting of Amazon's duplicity.
_____________________

Strive for progress. Not perfection.
_____________________

Tom

Quote from: Darren Dirt on May 12, 2014, 10:18:09 AM
imo you are far trusting of Amazon's duplicity.

amazon doesn't make duplicity. its by the guy that made rsync. I just use its s3 backend. and its compressed and encrypted so amazon couldnt look at it easily.
<Zapata Prime> I smell Stanley... And he smells good!!!

Mr. Analog

The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon
By Grabthar's Hammer

Tom

Quote from: Mr. Analog on May 12, 2014, 10:30:41 AM
The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon
Yeah, and theres a reason I have more than one remote destination. just in case, you know?
<Zapata Prime> I smell Stanley... And he smells good!!!

Mr. Analog

Quote from: Tom on May 12, 2014, 10:57:32 AM
Quote from: Mr. Analog on May 12, 2014, 10:30:41 AM
The content is encrypted before it goes to Amazon so really the trust is on the encryption. If S3 dies there'd be a lot bigger players than Tom with some expensive questions for Amazon
Yeah, and theres a reason I have more than one remote destination. just in case, you know?

That's the way to go yeah!
By Grabthar's Hammer

Darren Dirt

Quote from: Tom on May 12, 2014, 10:20:49 AM
Quote from: Darren Dirt on May 12, 2014, 10:18:09 AM
imo you are far trusting of Amazon's duplicity.

amazon doesn't make duplicity. its by the guy that made rsync. I just use its s3 backend. and its compressed and encrypted so amazon couldnt look at it easily.

1. Ah, makes sense how it works now
2. I was making a lame-ass wording joke  ::)  ;D
_____________________

Strive for progress. Not perfection.
_____________________