> RAID 10 a few NVME and you can get decent throughput (and storage size) with existing technology.

is there a good reason to do this in a consumer setup? max realistic throughput over gigabit ethernet is only ~120MB/s, which can easily be saturated by sequential reads or writes to/from a single modern spinning-rust drive.

My NAS has 3x 1gbit Ethernet ports

10Gb ports are quite cheap as well especially if you're only going point to point (the switches are still a little pricy, but the ports for a NAS to VM host are easily under $100 now).

Even switches like Mikrotik CRS305-1G-4S+IN are getting into 100 EUR range, (if you are fine with running optic cables instead of metallic).

Thanks!

I am fine with fiber; it's cheap and for a few ports at home, I don't worry about the power (especially as compared to the servers it cross-connects).

I wouldn't RAID10 anything, but it's totally feasible to build a ZFS RAIDZ2 pool with hotspare out of a whole bunch of 2TB SATA3 SSDs. People do similar for 'budget' video editing setups.

NVMe SSDs only significantly increase bandwidth, but not random throughput. Unless you have a 40gbit network you're not going to see the difference between one NVMe SSD or two if you're accessing them over the network.

I do everything baremetal, and my servers are a few extra (and quite decent) NICs for direct connection to eachother so that no even switches or routers bring their overhead

How much are you gaining by having multiple interfaces, with each one going to separate machine, instead of bonding them and using a switch?

Basically you dedicate 1/N to each server, instead of allocating it dynamically on demand.

I gain over 10% extra performance for what I use my servers for (with 100% meaning one full new server)

Latency is critical for some tasks, with bandwidth being a distant second.

I don't know that I'd consider RAID 10 for prosumer. RAID 1, absolutely. RAID 10 seems like needlessly adding more complexity and higher failure rates.

Great point. Complexity is killer in home setups.