NASs and Home Lab/Lan HW

[Tom]

I recently started working on fixing the problems I have with my server's raid arrays.

TL;DR: I need to plan some new home lab stuff. New storage arrays and/or a buy/build a new NAS and firewall.

Long story:

The array I used to backup the nas was dying for a while before I moved to the new place and died just before the move. But I couldn't really justify spending a bunch of money on that AND I didn't have the time/energy to deal with it.  For some reason I decided it was ok to run that as a raid0 of two 2TB Seagate desktop drives. I think I just had limited space for disks and since it was ONLY a backup, it didn't really matter if it died, so long as it didn't die at the same time as the NAS lololol. Guess I forgot how long I can delay basic maintenance.

The VM raid10 storage array also has a missing disk that I need to replace. It's made from old 2TB Seagate desktop drives. Both arrays made from the same disks pulled from my old /old/ nas setup.

I've been running bad blocks tests on some older drives I have to at least replace the NAS backup array.  I still have 3 3TB WD Reds I'm probably going to use for that.

And I just checked the nas and I noticed a bunch of medium errors in the dmesg log for the main nas array. No raid errors so far, apparently the disk has been doing its thing reallocating sectors (192 reallocated, and 1 offline uncorrectable/pending) behind the scenes and throwing the occasional error, and the raid has being doing its thing dealing with it, and all the while I haven't noticed a thing. Looks like I forgot to set up the smartmon emails on the nas the last time I rebuilt it. *sigh*

AND while looking at the disks, the NAS's root raid1 has a disk with a few reallocated sectors, but man those disks are fairly old 500GB seagate's. Old enough that the smart info power on hours counter has wrapped. So 7+ years of solid uptime. Might be my most reliable drives to date actually. But they also saw very little use iirc.

So now I need to come up with a good plan to rejig some stuff.

This time I think I should do it right. No el-cheapo consumer disks. Maybe blacks as ~6 years isn't TOO bad. But I think its better to start looking at enterprise gear. Even enterprise mechanical disks aren't that expensive anymore as long as I don't go for the highest capacity stuff.

And I'm thinking I may even get a stand alone NAS box for the backup. I will probably want something with a bit of beef for my main nas so I can run a few things (like nextcloud) without getting annoyed.

My nas is currently my /old/ home server hw after the nas motherboard failed (probably due to heat), so its running a Xeon E3-1230, on a Supermicro X9SCL with 16GB ram. Getting a bit long in the tooth.

AND I've been eyeing getting a new firewall appliance. My current Pc-Engines APU2c4 is not actually fast enough to handle my new internet fully, let alone run anything else while attempting it.

So I basically need to start from scratch. New new NAS, new firewall, ideally 10GbE capable hardware.

I'd apprecaiate some suggestions. I tend to get bogged down in analysis paralysis which is why I haven't even bothered to get new drives. Ideally I need something I won't have to think about too much. I just don't have the time/energy to spend on thinking about it let alone doing manual maintenance. Yet be something flexible and performant enough I won't get annoyed with it later on.

Hell, I've been looking at a solution for adding motion sensing to our front porch (pot) light, and been getting stuck thinking about it. Royal pain in my rear.

[Melbosa]

Budget and why 10gE?

[Lazybones]

Budget and why 10gE?

If I was starting over right now I would be looking at 10GE or at least 2.5/5 Gig capable devices..

If you are in a Fibre area you can already get 1.5 Gibit plans.. Not that you NEED them but it looks like greater than 1Gig is going to become relatively common "soon" in cities at least.

WiFi 6 APs are now shipping with 2.5 or 10Gig interfaces on them since in THEORY they can exceed 1Gig throughput.

However the router/firewall is probably the most expensive part however since doing inspection at those speeds is still a little bit of a premium.

[Thorin]

The first step to getting out of analysis paralysis is telling yourself that you're probably going to pick wrong no matter what, so there's no point in worrying about picking wrong, so just go ahead and decide and then do.

I use this technique when buying used cars - I assume I'm going to end up with a lemon so that I'm already in the right headspace if it needs thousands in repairs.  Now I can pick a car that I can tell isn't perfect and not beat myself up for it.

As for home network / NAS, unless you're actually making money off of it, there's no point in buying super-expensive high-end newest-technology stuff.  If it were me, a good Synology with a bunch of drives and Synology Hybrid Raid enabled is good enough, as long as any of the *important* files are backed up offline.  Gigabit Ethernet would be fine, too; sure, future technology might allow faster file transfers but it's already plenty fast and why pay extra for _possible_ futures?  A good LAN setup with extra APs covering low spots, I'd say that's much more important.  Make it so internet is fast in any part of the house including the garage.

Five years from now, when devices will probably be routinely capable of above-1Gb speeds, that's when you'll be looking to upgrade again anyway.

[Lazybones]

Did a little hunting and 8port CHEAP 10Gig switches are still quite pricy at $449.99 CAD.

2.5Gig capable switches start around $164.99 for a 5 port QNAP (QSW-1105-5T) 5 port 2.5Gbps

Cheap 1Gig switches are like $50..

[Tom]

Budget and why 10gE?

Budget? Hmm. Say a couple grand? No super hard limit as i can just buy things incrementally.

As for why 10G? Cause. And I've wanted to play with booting vms over ethernet via iSCSI or AoE/NvmeOE etc.

Did a little hunting and 8port CHEAP 10Gig switches are still quite pricy at $449.99 CAD.

2.5Gig capable switches start around $164.99 for a 5 port QNAP (QSW-1105-5T) 5 port 2.5Gbps

Cheap 1Gig switches are like $50..

I've got a couple decent netgear GbE switches. one 24 port and one 8+ port PoE. No shortage of ports.

MikroTik has some affordable 10GbE switches which I've been looking at. $136 for a 4 port SFP+ desktop switch, and $280 for a 8+1 SFP+ switch. So things have gotten pretty good. SFP pcie cards have also gotten "cheap". Its just buying the cables and/or transievers that can really add up.

As for home network / NAS, unless you're actually making money off of it, there's no point in buying super-expensive high-end newest-technology stuff.  If it were me, a good Synology with a bunch of drives and Synology Hybrid Raid enabled is good enough, as long as any of the *important* files are backed up offline.  Gigabit Ethernet would be fine, too; sure, future technology might allow faster file transfers but it's already plenty fast and why pay extra for _possible_ futures?  A good LAN setup with extra APs covering low spots, I'd say that's much more important.  Make it so internet is fast in any part of the house including the garage.

I've heard some stories about Synology and the like. I wouldn't want to depend on one of them for anything more than backups. If I'm going to spend the $1k on a decent sized Synology or the like, shouldn't I get something a bit more flexible?

Turns out my shaw modem's wifi covers the entire house from the basement corner "good enoguh" for now. So far its been less flaky and faster than my old /expensive/ Unifi AP-AC-Pro.

[Lazybones]

If I'm going to spend the $1k on a decent sized Synology or the like, shouldn't I get something a bit more flexible?

Went from building my own to a synology.. The Synology has been the least problematic but I don't use it as a server, I started out running some stuff on it but ended up migrating that out to a dedicated NUC with a more powerful CPU.

Pros of a Synology:

- Small
- Hot swap drive bays
- Low power
- Mature software
- drive failure detection works well if you are using supported drives. This is more important than you might think as it will pull drives as failed BEFORE complete failure sometimes based on drive data.
- Strong notification and remote access options
- Personally I use Surveillance station and while costly (per camera) it has been fairly solid software.

In short less dicking around

Cons:

- More expensive
- Lower compute power if you want that combined
- gets pricy for larger units
- some arbitrary limitations

Not for everyone but as long as they don't force their own drives on the consumer units like they have for the business / rack units I will probably upgrade to another synology.

[Mr. Analog]

Seconded on the Synology front. For the price I paid I'm very happy with mine. It's easy to use and has a boatload of features that are easily accessible

Sent from my Pixel 2 XL using Tapatalk

[Melbosa]

FYI: As for why 10G? Cause. And I've wanted to play with booting vms over ethernet via iSCSI or AoE/NvmeOE etc. - Works fine over 1gE - do this at home lol.

[Lazybones]

FYI: As for why 10G? Cause. And I've wanted to play with booting vms over ethernet via iSCSI or AoE/NvmeOE etc. - Works fine over 1gE - do this at home lol.

On this note iSCSI configuration is quite easy on a Synology, as is NFS if you prefer that for doing something similar.

Back on the 10Gig note I would note that even for me I would not consider it necessary for more than a few CORE switch ports and that for most connections 1Gig is fine..

[Melbosa]

Back on the 10Gig note I would note that even for me I would not consider it necessary for more than a few CORE switch ports and that for most connections 1Gig is fine..

Yeah, I would only consider this for my upstream ports and connection to my gateways/routers (so we can get more than 1g Inet service).  But really not needed for any other services I have, and I have heavy server farm deployment in my house with 50TB Synology 1812+.  I do however have a 40gE connection between my two compute heavy servers, BUT the only time it is used is in v-motion activities.

[Melbosa]

So to swing back to your original request of: New new NAS, new firewall, ideally 10GbE capable hardware - but what I think you really mean in your op is:

• New Server with Storage - compute and storage - primary
• New NAS - backup
• New Firewall/Gateway

If that is correct, this is my recommendation:

• Server + Storage: A gently used server from a corp offloading - I have had great success with https://forums.servethehome.com/index.php - particularly buying form guys like https://forums.servethehome.com/index.php?members/patrick.1/ who works for a cloud provider (or at least he did). I would target something that can still take regular 3.5 form factor drives and with a SAS controller so you can basically put any drives you need to in it.
• NAS Backup: Love the synology if you don't need the compute for the storage. If just for IOPS and conductivity to a storage array, can't beat them - I even have some of their rack stuff in the wild at client's sites and they are just as solid
• New Firewall/Gateway: Personally I'm running a PFsense Netgate Appliance: https://www.netgate.com/appliances, and at my client sites (which I equate to what you would need for your house as well if not looking at fore mentioned) I use WatchGuard or Ubiquiti.

[Melbosa]

Synology on sale today at MemEx: https://www.memoryexpress.com/Category/NetworkNAS?FilterID=cdd11fc5-f5be-cc42-7d2a-2b4c34738c47&Search=MX0011303

[Lazybones]

Canada Computers also has several NAS on Sale

https://www.canadacomputers.com/index.php?cPath=1035&sf=:&rebate=1&mfr=&pr=

[Tom]

Well crap. I had just written a nice post and chrome locked up and thinking I was just going to kill one window, I let the OS kill the process... except it was the entire master chrome process that was locked up so it killed all the chrome windows. Sigh.

At any rate, I like the suggestions. I'll check out the links. And I wish I had seen the sale links sooner. that DS920+ seems about what I want for a temp main NAS and eventual main backup NAS.

Speaking of my NAS, on the 22nd I re-setup the SMART tests. On the 24th, the drive with the errors fully died. Sigh. I just noticed since It appears I'm not checking my email enoguh. I need to somehow prioritize those emails/notifications better. So Now my main nas is running fully degraded and it may only be a matter of time before more die as they are all likely from the same batch. Though none of the other WD blacks are showing bad sectors, they are all showing 50k+ power-on-hours (5.7 years). Much of that is 25/7 on, which is far more than they are rated for IIRC. Out of warranty last Feb 1st. Had to check lol.

Not a super fan of netgate/pfsense these days for various business/ethical decisions they've made the past few years. My oppinion of Ubiquity as dropped lately as well. But we'll see. I may just grab one of their firewall/controller boxes since they are quite nice and the interface is pretty nice. We'll see.

FYI: As for why 10G? Cause. And I've wanted to play with booting vms over ethernet via iSCSI or AoE/NvmeOE etc. - Works fine over 1gE - do this at home lol.

On this note iSCSI configuration is quite easy on a Synology, as is NFS if you prefer that for doing something similar.

Back on the 10Gig note I would note that even for me I would not consider it necessary for more than a few CORE switch ports and that for most connections 1Gig is fine..

Yeah. I mostly just /want/ 10Gb between the firewall, my server/nas(s) and my workstation. Everything else will be 1Gb or wifi. Note, its not a NEED, just a want. I've had some bottlenecks I'd like to reduce, in one way or another. 10Gb would help there, as would shuffling around services a bit. Its also just something that'll be fun to play with.

Like my nextcloud has historically been slower than I'd like. It's improved over time, as I've tweaked things, and nextcloud itself has improved in performance. But I'd like to either get the network connection BEEFed up, or just move it onto the nas. I've not yet done either as it's just another TODO item on my seemingly infinite TODO list.

Having A LOT more space available to my build server would also be nice. An AOSP build gets really large. And storing the artifacts for them as well as my other projects takes up enormous amounts of space. And I'd like it to not be slow. Some of the artifacts are in the multi GB size. Because of all of this, I've mostly just been not using the build server for AOSP work at all, and just adding more ssd storage to my workstation. Just for linux I've got a 500GB+1TB nvme for / and /home, 2TB nvme for ~/build, and a 1TB sata ssd for ~/bulk which mostly stores older AOSP checkouts that I don't want to get rid of yet, and also don't want to put on a HDD since I'd like searches through it to go relatively quickly. Even with all of that, I'm still running out of space on ~/build. lol. I keep getting new devices to either to move stuff from ~/build onto, or larger ones for ~/build and ~/build keeps filling up. Sigh. Got all the slots filled in my workstation too. Even with the pcie->nvme card I grabbed. Sadly it doesn't have a pcie switch on it so only two of the four nvme slots on the pcie card work with my motherboard.