NASs and Home Lab/Lan HW

[Lazybones]

Yeah. I mostly just /want/ 10Gb between the firewall, my server/nas(s) and my workstation. Everything else will be 1Gb or wifi. Note, its not a NEED, just a want. I've had some bottlenecks I'd like to reduce, in one way or another. 10Gb would help there, as would shuffling around services a bit. Its also just something that'll be fun to play with

Going 10GB on a Synology is somewhat expensive most of the ones that support it are much higher price / larger units. Unofficial options you need to be careful with as DSM 7 removed some unofficial drivers.

Routers are another issue, most common suggestion seems to be to go custom with PFSense, however it is possible to get some EdgeRouters and a few others that are capable but at a premium.

Switches are not that hard.

I would not bother going 10GB or 2.5/5G unless you can actually implement at least 2 of the 3 objectives.

Also keep in mind figure out how YOU want your environment to work, our group is fairly diverse in terms of budgets, patience, technical experience, and personal time. My suggestions stem from not wanting to dick around and having a bit more money to spend.

[Tom]

Yeah. I mostly just /want/ 10Gb between the firewall, my server/nas(s) and my workstation. Everything else will be 1Gb or wifi. Note, its not a NEED, just a want. I've had some bottlenecks I'd like to reduce, in one way or another. 10Gb would help there, as would shuffling around services a bit. Its also just something that'll be fun to play with

Going 10GB on a Synology is somewhat expensive most of the ones that support it are much higher price / larger units. Unofficial options you need to be careful with as DSM 7 removed some unofficial drivers.

I probably would be fine with 1GB on a synology as its probably going to end up being a backup. Depends on how annoying it would be long term.

Routers are another issue, most common suggestion seems to be to go custom with PFSense, however it is possible to get some EdgeRouters and a few others that are capable but at a premium.

Switches are not that hard.

I would not bother going 10GB or 2.5/5G unless you can actually implement at least 2 of the 3 objectives.

Also keep in mind figure out how YOU want your environment to work, our group is fairly diverse in terms of budgets, patience, technical experience, and personal time. My suggestions stem from not wanting to dick around and having a bit more money to spend.

Yeah. I /want/ to not dick around, but I equally don't want to be too limited. Its a conflicting set of desires.

[Lazybones]

Yeah. I /want/ to not dick around, but I equally don't want to be too limited. Its a conflicting set of desires.

This is why I ended up moving all the computer off my NAS on to an Intel NUC running Ubuntu and Docker. Synology has solid BTRFS storage and backup. The NUC allows me to run anything I want. It is also similar to the separation I would have in production systems at work.

I found it easier and more reliable than dedicated storage OS distributions as well at least in terms of it quote "JUST WORKS" with the hardware.

[Tom]

So because "budget", the grand plan is being split up into phases.

Given the NAS has a drive down, and it contains a lot of stuff I use every day and a lot more stuff I'd hate to lose even if I don't use it very often, I ordered 4 14TB Exos X16 drives for $366/ea (~$1540 after tax all in). Going to set them up today and run some burn in tests to (hopefully) ensure they wont up and die right away. I've had ASTONISHING luck with HDD infant mortality. I've learned its worth the time even if it takes DAYS to run the tests. It will probabably take 8-10 days to run. A single badblocks run on my old 3TB WD Reds took 57 hours! Haha. So I'm going to run them in parallel. I did the Reds one by one on my desk with a usb to sata dongle.

I ALMOST got a Synology, either a DS920+ or a DS1520+. Based on the price and features difference I was heavily leaning towards the DS1520+ then I decided, no, I should just start with ensuring I don't lose the files I've already got. hah. I was also tempted by some of the Asustore NASs as they kept getting good reviews and typically have better hardware than the synologys but I dont think their OS is as popular for addons and support.

Anyhow, I'm about to double my storage, even though I don't need to. I just wanted to get the best bang for the buck. So I did a bunch of googling and sorted drives by $/GB, then prioritized NAS or enterprise drives, then kept the price per drive under like $400 so I can justify buying enough drives for a decent raid array. hah. 3 was my absolute minimum but I wanted at least 4. Thinking about it now, I might run raid6 since I really don't need the space. But then thats $700 in drives used as parity. lol. so perhaps not. Hell If I did that I should probably just go md-raid10. Hey, I might just test all three modes and see what performance is like. a raid5 vs raid6 vs raid10 shoot out with my specific use cases.

Once I get these drives installed and started testing, I'll migrate my VM box's vm array over to the WD blacks that are left, and I'll recreate the old backup array with my older 3TB WD reds.

After that's all done, I think I'll look at some used enterprise hardware for the VM box, like some sun f80 or similar flash cards, some used network cards and switches etc, and perhaps one of the ServeTheHome "Project MiniMicro" picks for a new firewall.

Then perhaps, depending on need, I'll either build a new vm box or nas. Though to be honest, I might actually just go with a synology as the next big purchase so my important files have a decent second home.

I've gotten hesitant of appliances having more than one main task. Like back in the day I ran everything on one single box, file server, services/vms, and firewall. doing maintenance on it took everything down at once. So I split things up a bit, got a separate firewall so if I need to do maintentance on the server, the internet doesn't go down! Then I got tired of maintenance on service related stuff taking down the nas, and nas related stuff taking down services, so I split those out to separate machines. So yeah. Fun.

Yet another one of my long winded yaks. Sorry. I try to keep these things short but my brain likes to ramble. And re-writing it would take up valuable work and system maintenance time. Believe it or not though, I do trim and rewrite as I go to keep it somewhat sane-ish... haha.

[Tom]

God man.. I tell you what.

So I'd been running tests on the new drives. One started throwing CRC errors, but things kept going. The errors typically meant a bad cable or some similar problem, but after fiddling with the cables a little nothing helped. So I stopped the tests, and moved the disks arround a little...

And that caused one of the main nas raid disks to drop. in a raid5 thats already got a missing disk. *sigh*. Things are fine now, but it took some (mostly) careful work to fix. Ended up having to re-create the array over top the old. it mounts and xfs_repair runs successfully. Doesn't appear to be any errors.

That one new disk still seemed to be throwing errors, so I swapped the sata cable with another of the new drives, and now no more crc errors? I don't even. Perhaps a dirty contact on one of the sata pins? and a reinsert cleaned that up? W/E.

Re-running the badblocks tests now. May take ~5 days assuming each of the 4 write/read pass pair goes at about 250MB/s average.

Since I'm using a SAS card, I have to use those not cheap sas->sata breakout cables, so I was hoping it wasn't a problem with the cable.

Makes me think though, the first failed disk used that same cable. The errors appeared to cause the disk to want to do resets, so I wonder if that can cause a drive to get bad sectors? Werid.

[Tom]

After a little cable jiggling and reinserting, I'm not getting the random drop outs. So I think it was a combo of loose cables and potentially dirty connections.

I've coppied all of the data off the old array I can onto my old WD Reds, not enough room for the anime so thats the only files I'm going to risk losing during the next step.

At any rate, gotta finish building the new nas array. And I've learned OpenMediaVault has finally released an update. Prior to i think a few months ago it was still based on Debian 9, originally released in 2017. Yow. new version looks to be based on Debian 11. Released in 2021. so much more recent. And its got a new web ui, so maybe it'll suck less, but i think it will probably have fewer addons, but hopefully won't need as many of them? So I'm either going to upgrade, or think about starting fresh with TrueNas Scale (or Truenas Core aka FreeNas), Or UnRAID.

In other news, I got a new firewall. Ordered it barebones. Just waiting on ram to show up. I wanted to avoid buying whatever pos ram/storage they'd ship with it just to replace it later. Its fanless, with a huge heatsink. Claims 15W power use. Quad core Intel 11th gen I5 mobile/embeded cpu. will be putting 16GB ram into it and a couple sata devices from my old firewall. 6 built in GbE ports, I'm hoping its got more than one controller and not all sharing the same one on a switch, so it could theoretically handle more than 1GbE. According to the specs, it says "6 intel gigabit network cards, support 5 same broadband access," so I'm thinking that means two actual nics, with one port with a dedicated nic, and 5 ports sit on a switch chip. That to me is acceptable.

Its going to have significantly more beef than my old firewall. Considering the old boy has issues loading in the pfsense ui let alone handle my new 1Gb/100Mb internet. I normally get like 600-700Mbps down and 100 up, but the cpu use spikes over 80% on down. When it does get to over 800Mbps it hits 100% cpu. This is after disabling a bunch of services and any kind of limiting, queueing or fancy filtering/blocking. Direct to the modem my desktop gets > 900Mbps. Fun.

[Lazybones]

Are you planning on running pfSense or something else?

[Melbosa]

At any rate, gotta finish building the new nas array. And I've learned OpenMediaVault has finally released an update. Prior to i think a few months ago it was still based on Debian 9, originally released in 2017. Yow. new version looks to be based on Debian 11. Released in 2011. so much more recent.

LOL I think you mean 2021 not 2011

[Lazybones]

At any rate, gotta finish building the new nas array. And I've learned OpenMediaVault has finally released an update. Prior to i think a few months ago it was still based on Debian 9, originally released in 2017. Yow. new version looks to be based on Debian 11. Released in 2011. so much more recent.

LOL I think you mean 2021 not 2011

It's debian, I didn't even question if 2011 was considered NEW for a stable release LOL.

[Mr. Analog]

LOL

Sent from my Pixel 2 XL using Tapatalk

[Tom]

At any rate, gotta finish building the new nas array. And I've learned OpenMediaVault has finally released an update. Prior to i think a few months ago it was still based on Debian 9, originally released in 2017. Yow. new version looks to be based on Debian 11. Released in 2011. so much more recent.

LOL I think you mean 2021 not 2011

You're right lol! I typo things up normally then I revise and re-edit and different errors sneak in lol.

Are you planning on running pfSense or something else?

Probably opnSense. I'm not super happy with pfsense these days. nor with netgate itself.

[Tom]

New firewall has been working reasonably well. Opnsense isn't quite as user friendly as pfSense, but it does have wireguard and a little more customizability.

I also recently set up the old UAP-AC-Pro I've had for a while. The new unifi network manager software apparently has a default setting to auto update APs even though they should know they have issues with releasing dodgy firmware and forcing people to update by default is probably a bad idea. That said the new firmware appears to work rather well compared to the old ass fw I was on. Not having the same range issues I was in the trailer. Actually got signal upstairs from the /basement/. Though it was a bit flaky two floors up. Still A LOT better than being flaky 30 feet away with no real obstructions.

Got a Unifi U6-Pro when a "in stock" notice I set up recently fired off. So far they are working pretty well. Got the old AC-Pro in my office and the U6-Pro in the master bedroom temporarily.

I wired up the 3 cat5e lines they ran for phone lines with ethernet jacks so I could plug my hue hub in on the main floor so it can talk to our front light and motion sensor, and so the wifi can plug in upstairs till I run some dedicated AP lines.

Have to say most people's APs are freaking dumb. Most are all huddling on channel 6, probably because one or two networks are set to 40mhz? Like seriously, someone is hogging most of the spectrum with using wide channels and the rest of the networks are all sticking to the same channel. Even my APs seemed to want to select channel 6 when I did a channel scan/optimize. weird.

[Lazybones]

Have to say most people's APs are freaking dumb. Most are all huddling on channel 6, probably because one or two networks are set to 40mhz? Like seriously, someone is hogging most of the spectrum with using wide channels and the rest of the networks are all sticking to the same channel. Even my APs seemed to want to select channel 6 when I did a channel scan/optimize. weird.

Could be worse you could have multiple  APs NOT on 1, 6, 11 interfering with basically everything and making 2.4Ghz almost unusable for everyone.

The joy of living in a townhouse super close to other people

You cannot view this attachment.

[Melbosa]

We've noticed (my company partners) that DLINK routers are bandwidth hawgs something terrible... using our site survey analyzer as of late at a lot of our client sites, and we can find home DLINKs eating up significant bandwidth and saturating the airways in the 5G space.  Causing terrible issues for our clients.

No fault of a home user, they don't know any better.  Seems to be the default setting of DLINK's wireless settings on their latest routers!

[Tom]

Heres what I'm talking about:
You cannot view this attachment.