When Gigabit isn't very fast anymore

Lazybones · October 13, 2014, 07:03:28 PM

I went do do some moving around of my home configuration this weekend and realized two things..

1. 3TB of data is a lot to move around on consumer gear
2. Gigabit network speeds do not make moving around TB of data very easy it takes HOURS.

I guess you get spoiled moving data around on 10Gig network on SANs containing over 100TB of space when you work with it every day.

Thorin · October 13, 2014, 08:39:38 PM

One day I'll upgrade from 100Mbit/s...

Lazybones · October 13, 2014, 09:10:35 PM

Went and crabbed a 4TB USB3 external drive on sale to backup some of the data so I could reformat the drives.. The throughput is still only about 107MB/s.. which isn't too bad for a single drive.

Darren Dirt · October 13, 2014, 11:27:26 PM

Quote from: Lazybones on October 13, 2014, 09:10:35 PM
Went and crabbed a 4TB USB3 external drive on sale to backup some of the data so I could reformat the drives.. The throughput is still only about 107MB/s.. which isn't too bad for a single drive.

I've been thinking about doing similar for my home data -- mostly text and some MP3s and images, very little video. Right now moving EVERYTHING onto my laptop HD but realizing it would totally suck to lose all my data -- what's the reasonable price range of a 1 or 2 TB USB nowadays? And is it all SSD now (for external USB drives)?

Was also thinking about throwing an additional copy of the most critical stuff onto The Cloud, but don't know where to start looking. Dropbox even offers 1 TB for $10.99 monthly (or $109 yearly)! https://www.dropbox.com/upgrade ... heck that's cheaper than my non-basic NETFLIX cost

but idk if that's over-priced compared to others (i.e. taking advantage of name recognition + lazy customers) so might want to look around but i gotta do *something*...

Lazybones · October 13, 2014, 11:35:15 PM

My Primary data is on a RAID made up of 3 x 3TB drives in a RAID 5 (yes I know the rebuild time could be too long).

The external single drive I picked up was the WD My Book 4TB USB3.0 for $159 on sale.

FYI My primary transfer is still running (ETA for coping 3TB directly over USB 3 was 11 hrs)... My project is on hold till I move all the data.

Oddly internal drives that size are the same price or more.

I figure once I juggle things I will keep a potion or a copy of the working data on the external drive in-case i have a big failure.

Media files I can't replace such as my photos I keep locally and on free online storage like Google Plus.... I have several Gigs of photos now as my cameras keep getting higher resolution.

Mr. Analog · October 14, 2014, 08:25:20 AM

I think the way to go is reduce latency and increase bandwidth with some removable SSDs and use sneakernet to transfer large files across your home network.

Since I got my new gaming rig I've wanted to start pushing data around the house and realized the fastest way for me to do that was to just buy a few drive docks and move stuff around that way. Now it should be said that I can saturate my own network as the single user with device crosstalk even doing nightly file transfer batch runs (my network game is weak) so when I had this idea last month I was really surprised at how well it worked / how little fuss it actually was.

Also doing it this was is slightly less of a blind copy across disks I started looking at some of the digital baggage I was carrying along and started to mark large swathes of it as redundant (after making copies of course, just in case). For absolutely critical stuff I actually dusted off my DVD-R spindle (which I think I bought 10 years ago lol).

Anyway, that's what helped me out, a few TB drives and some docks.

Lazybones · October 14, 2014, 09:15:33 AM

As I already stated I gave up on network transfer and went to a USB 3 attached external drive.. however I am pushing the single drives performance at this point and only moderately faster than Gigabit.

Mr. Analog · October 14, 2014, 09:19:10 AM

I see yeah, it's still fairly slow, that's for sure

Thorin · October 14, 2014, 11:09:24 AM

Quote from: Lazybones on October 14, 2014, 09:15:33 AM
As I already stated I gave up on network transfer and went to a USB 3 attached external drive.. however I am pushing the single drives performance at this point and only moderately faster than Gigabit.

Yeah, USB3's theoretical performance of 480MB/s is way, way higher than actual observed speeds of 80-100MB/s...

Lazybones · October 14, 2014, 11:15:31 AM

Quote from: Thorin on October 14, 2014, 11:09:24 AM
Quote from: Lazybones on October 14, 2014, 09:15:33 AM
As I already stated I gave up on network transfer and went to a USB 3 attached external drive.. however I am pushing the single drives performance at this point and only moderately faster than Gigabit.

Yeah, USB3's theoretical performance of 480MB/s is way, way higher than actual observed speeds of 80-100MB/s...

It isn't the USB3 that is the bottle neck it is the spinning hard drive attached to it... the CACHE on the drives will let you burst up to those limits when you are lucky and doing lots of repeat reads... However it doesn't do a whole lot for continuous large transfers.

A HIGH end SSD is needed to saturate USB 3 or SATA 6Gibit interfaces for the most part. You aren't going to find 4TB of that for cheap yet.