Maximising or improving home network speeds/performance?

[neonplanet40]

Hi folks, I am due to get an upgrade to FTTP internet, and this led me to have a closer look at my current home network. I have tried to illustrate it as best I can. One of the wireless access points (the top right) are based upstairs. My Wi-Fi is provided by the two Asus routers as a mesh network and this was just recently upgraded to these two new routers. I don’t have any 2.5GBE equipment, so for me it is about maximising the 1GBE speeds across the network as best I can. One thing I forgot to add was another Asus router (ac68u) which has its own wifi network. However, this is only used as a backup should the mesh fail (as has happened once or twice).


I currently transfer a lot of files to my NAS (TB’s) and I am keen to maximise my speeds. I get around 60-70 MB/sec. If I could get this higher it would make a difference for me. I am worried that maybe my network has too many switches (if this is possible) OR I could have a potential bottleneck (with too much going through one switch that could perform slightly better if I split it over 2?). Could adding more switches to try and balance the load a bit further help? And if so, where would be best to add them? I have 2 spare D-link GB switches sitting around, hence my question .


In the future, when I upgrade NAS etc I will likely make a move to 2.5GBE or higher if realistic. But 1GBE is the norm for now. It is not sometime I am doing right now.


However, I thought asking others more knowledgeable than me could have a look and provide some help and feedback? While I am working on my network, I just wanted to do what I can now given that it is already something I am going to be looking at as my internet gateway will be moved (as the fibre entry point is not the same as my current copper entry point).


Thank you for your help and I hope it makes some sense!

An illustration of my network is in the link below. When I get home, I will host the picture (work blocks personal storage etc etc so unable to do it here) and put that up instead.


https://ibb.co/T004CvL

Thanks for your help folks.

[DanceswithUnix]

Sorry, but I don't have an account on lucid and don't fancy yet another source of spam emails. If I had seen the other thread earlier I could have suggested "dia", but a bit late now.

Anyway, some general observations on what you said...

You can have quite a few switches in a chain, something like 5 without issue which is probably more you are likely to get in a home environment. Note that is traversed not total, so if you have one central switch that feeds switches in other rooms in a star, then packets will only be crossing the room switch where they start, the central switch, and the switch in the destination room. That leaves wiggle room for a couple more switches badly daisy chained in without problems.

Adding more switches generally doesn't balance loads, the switch just finds what it thinks is the best path and sends everything down that one, if you are lucky. If you are unlucky, you get a loop which causes a network storm and everything grinds to a halt hopefully just for a short time until the switch disables one of the redundant connections.

If you want more throughput, you need link aggregation to turn 2x1GbE into a 2GbE connection which means paying for managed switches. That can work, but isn't ideal.

Transfer performance from a NAS can depend on a lot of things. I run PCs has home servers, and am currently migrating from an old FM2 quad core machine to one based on a Ryzen 3600. Both use a pair of hard drives for storage. Transferring a couple of VM images from the old server to the new one, the first one I used sftp from the new server to read the file from the old one, and got 70MB/sec transfer speed. Thinking that was a bit disappointing, the next one I logged into the old server and transmitted the file to the new one, getting 100MB/sec. Both big files, both same actual direction, but just down to whether I was pushing or pulling the data. Network protocols can be like that though, even though even my old FM2 Athlon is way more powerful than most consumer NAS boxes and the new server has an Intel network card, so play with your workflow.

OFC the smallest of those files was over 11 minutes to transfer, so the real answer is to upgrade to 10GbE or better. That means parting with money though, not something I'm good at

[neonplanet40]

I think I may have put a link up now to use so that lucid doesn't need to be joined (I had though the link would allow people to view it without joining... grrrr).

the img bb codes aren't working with the link so I just added a direct link to the image instead.

[neonplanet40]

Sorry, but I don't have an account on lucid and don't fancy yet another source of spam emails. If I had seen the other thread earlier I could have suggested "dia", but a bit late now.

Anyway, some general observations on what you said...

You can have quite a few switches in a chain, something like 5 without issue which is probably more you are likely to get in a home environment. Note that is traversed not total, so if you have one central switch that feeds switches in other rooms in a star, then packets will only be crossing the room switch where they start, the central switch, and the switch in the destination room. That leaves wiggle room for a couple more switches badly daisy chained in without problems.

Adding more switches generally doesn't balance loads, the switch just finds what it thinks is the best path and sends everything down that one, if you are lucky. If you are unlucky, you get a loop which causes a network storm and everything grinds to a halt hopefully just for a short time until the switch disables one of the redundant connections.

If you want more throughput, you need link aggregation to turn 2x1GbE into a 2GbE connection which means paying for managed switches. That can work, but isn't ideal.

Transfer performance from a NAS can depend on a lot of things. I run PCs has home servers, and am currently migrating from an old FM2 quad core machine to one based on a Ryzen 3600. Both use a pair of hard drives for storage. Transferring a couple of VM images from the old server to the new one, the first one I used sftp from the new server to read the file from the old one, and got 70MB/sec transfer speed. Thinking that was a bit disappointing, the next one I logged into the old server and transmitted the file to the new one, getting 100MB/sec. Both big files, both same actual direction, but just down to whether I was pushing or pulling the data. Network protocols can be like that though, even though even my old FM2 Athlon is way more powerful than most consumer NAS boxes and the new server has an Intel network card, so play with your workflow.

OFC the smallest of those files was over 11 minutes to transfer, so the real answer is to upgrade to 10GbE or better. That means parting with money though, not something I'm good at

Thank you for your reply. Does my network layout change any of your recommendations?

I would like to move to a faster network speed (Like 2.5GBE etc). But that's really a no go until my Synology is replaced. And Synology have been pretty poor on moving on from 1GBe ethernet. And no doubt when they do, the move for me to a new 12 bay NAS, upgraded switches and network cards etc will be a very expensive move indeed.

If what I have now is the best I can do, then this is fine. I just wanted to make sure

[DanceswithUnix]

Thank you for your reply. Does my network layout change any of your recommendations?

Nothing leaps out. The performance path you care about from PC to NAS has just two switches in the way, shouldn't be a problem.

But yeah, networking gear beyond the basic 1GbE rockets up in price. I'm starting to see at least the 2.5GbE ports turning up at a decent price, my wife's motherboard has one, and I see tp-link and qnap now have 2.5GbE consumer switches so if they are getting involved then it looks like these things are going mainstream. Fingers crossed the price comes down.

[Saracen999]

....

But yeah, networking gear beyond the basic 1GbE rockets up in price. I'm starting to see at least the 2.5GbE ports turning up at a decent price, my wife's motherboard has one, and I see tp-link and qnap now have 2.5GbE consumer switches so if they are getting involved then it looks like these things are going mainstream. Fingers crossed the price comes down.

I'm in the market for a NAS right now, and about the only things stopping me are that I had settled on Synology, but they're stuck on 1Gb and QNAP (for a comparable unit) are on 2.5Gb, and Synology only seem to allow NVMe M.2 SSD's in cache mode whereas QNAP (and Asustor) leave the choice between cache or direct storage to the user. Then again, Synology Hybrid Raid (SHR) really appeals to me, but so does Btrfs (on the QNAP).

Arghhh.

Also, I prefer the look of the Synology over QNAP.

[DanceswithUnix]

Arghhh.

This is part of the reason why I just self built a Linux box in a home theatre case. But then part of that is wanting to play with server configuration. That does of course lead to more choices though. Even if you decided to go with say a TrueNAS box, you could either run that natively on the machine or you could use Proxmox to create a one machine private cloud with ZFS storage and then make a NAS VM on that. But then with a VM farm, you can just try things out and delete the VM if you don't like it.

Some deja-vu here, good job the NAS doesn't have to make coffee as well

I suggest you go look at the price of 10GbE switches, then the 1GbE NAS won't seem so bad!

[Saracen999]

....

Some deja-vu here, good job the NAS doesn't have to make coffee as well

Oh, don't. It looks like my 3-month old Nespresso machine (the stopgap while I sort out water) died over the weekend, with a graunching noise and refusing to pump water.

I did think about TrueNAS but really, the whole point of buying the box is much, MUCH closer to buy, plug in and forget. I really don't want to have to faff about learning stuff, then configuring, tweaking, rebuilding (even in a VM). etc. Of course, it requires a degree of learning to try to sort out which pre-built box will do what I need, and then getting the maximum blend of what I want, because I can't get it all. So .... work out my priorities. For example, I would like SHR, but really, it's not do or die. I prefer the look of the Synology but, well, who really cares?

As for 10Gb, it seems costs have come down a LOT, but it's still not a cheap way to go. 2.5Gb switches, etc, aren't thick on the ground but there are a few, and not at totally silly prices either.

Again, it comes back to want vs need. Do Iwant 10Gb? Oh, hell, yeah. Why? Kinda like why people climb Everest .... because it's there. But do I need 10Gb? About as much as I need to climb Everest. I mean, seriously, no I don't need it. It'd be nice and I'm tempted but .... nah. In reality, I probably don't even have much of a case for arguing I need 2.5Gb.

Do you remember my stance on SSDs when they were fairly new? It was (largely still is, but prices have dropped a lot since) .... yes they're nice, fast, blah blah but do I need them? Will that extra bit of speed make any real difference to my workflow? Nope. And now I'm effectively retired, it's even more so. What on earth will I do with the extra few seconds, or very occasionally, minutes, that even 2.5Gb will give me? And should I care? No. I want it quick because, well, I want it but I really, if I'm honest with myself, don't actually need it.

What I do need is a decent coffee machine. That works for more than 5 minutes.

[DanceswithUnix]

And now I'm effectively retired, it's even more so. What on earth will I do with the extra few seconds, or very occasionally, minutes, that even 2.5Gb will give me? And should I care? No. I want it quick because, well, I want it but I really, if I'm honest with myself, don't actually need it.

The problem is the world changes and things get left behind. ISTM this has already happened with HDD vs SSD. I believe SSDs are now essential thanks to the sloppiness of many of my fellow software engineers.

Back when everyone had a HDD, if you wrote code that thrashed the head then it hurt you and you fixed it. These days programmers (including the people at Microsoft writing Windows updates) have an SSD, so if they write bad IO code then it doesn't hurt them. Worse still, if they are checking things like boot time metrics then multi threaded code that works best on an NVMe SSD will utterly tank anyone left with a HDD. Because that's hurting you and not immediately the dev, I was seeing hard drive thrashing more and more to the point I gave up.

HDD is the new tape drive, something that a consumer puts in a NAS box or video recorder. We have a handful of HDDs for bulk storage at work, but even our little QNAP has a couple of 1TB SSDs in it.

We now have WiFi standards and up coming home broadband that can exceed or at least saturate 1GbE connections. USB is hitting 20Gb/s. Networking is just so dated at this point, mainly because most people don't want wires let alone fibres.

Bad luck on the coffee machine, that really sucks I presume it can be warranty replaced, but must be annoying.

[Saracen999]

HDs still have their uses, mainly where the sheer bulk of storage needed makes SSDs prohibitively expensive. In my case, that's mainly video storage plus a few other bits, but it's why I'll probably go for a 6-bay NAS - 4 for large HDs(probably on cost/TG basis, 12TB's), 2 for future expansion, and a couple of 1TB M.2s with SSDs for more modestly sized stuff that I want quicker. Which is why Synology restricting their built-in M.2s to cache is a pain, and forcing me off Synology as a brand.

I wouldn't put HDDs with tape drives quite yet. But heading that way.

I think the coffee machine might have just been insisting on a clean and descale, which iy has now had. It seems to be working again. I suffer here from water hardness of the same grade of hardness as battleship main armour plate. Time will tell how long is resuscitates for though. It was making a funny noise. Funny (peculiar) not funny (HaHa). Oh, and I have an updated Wifi 6 router (Asus) currently in my shopping basket, as well as a laptop Intel "Killer" wifi 6 chip.

[amjad175]

I upgraded part of my network to 10GBE for FreeNAS recently, raw speed tests with tools like iperf look amazing but there is a bottleneck which I hadn't considered and I think you're overlooking as well, spinning HDD speeds and i/o capacity.

This is where the major bottleneck is going to be, doesn't matter if you have FTTP or 10GBE LAN. Next time you're doing a big read/write to the NAS check drive stats to see i/o capacity, if it's already maxed out then upgrading the network is not going to do much.

Regards.

[Saracen999]

What amjad ^^^ said.

My thoughts were that there's two main areas to consider. Access to internet, and LAN traffic. Something is always going to be the weakest link in each. In my case, the internet connection is okay, for my needs (maybe even more than I need) for anything I do but that isn't going to much change with 10Gbs on the LAN. So, no case for it there.

On the LAN, while occasionally I do enough transfers accross the LAN for 10Gbs to be much faster, it is not the general situation and if it saves me 5 mins, even half an hour, once a month, then it is still not a good value for money case. Even if going SSD to SSD (like Surface Pro to SSD on a NAS), and eliminating the HDD throughput bottleneck, it still doesn't justify (to me) the cost of going 10Gbs. Not yet, anyway.

What might justify it is, for example, several users all trying to watch 4k video simultantously, or direct-editing of 4k video stored on a NAS. Things like that. But there's only two of us here and rarely do we even want two different 4k streams at the same time. And frankly, I'd happily just watch HD or even SD TV and not bother with 4k at all.

So I think the case for 10Gbs LAN depends entirely on :-

- cost, which is coming down but still not necessarily trivial
- can you lay cable yourself (I can't, these days)
- what other bottlenecks (like HDDs) are there?
- the usage types on your LAN. Will you ever saturate 1Gbs, and if so ...
- Is 2.5Gbs, or even 5Gbs, a sufficient, and better value alternative?

In my case, it came down to "want it" but don't really have a justification for where the real world benefit comes from. Do I want it enough to just do it anyway? Not really, no. I'm still tempted, but no. YMMV, either in degree of justification, or in future-proofing, or even in just "because I can" factor which, to be honest, if you can afford it is all you really need. It does mean that, once done, it'll be a good, long time (if ever) before you need to think about upgrading again which might not be the case with 2.5 or 5Gbs.

If I were either building a house, or doing a large renovation project (that meant carpets, floorboards etc. were up anyway), then even having no real need, I would stick in the fastest network infrastructure I conceivably could, and I mean at least 10Gbs fibre, just so I hopefully never have to worry about it again. And for the tech-savvy buyer, it's a selling point if you ever sell the house. But I'm not, and the cable-laying needs is the major drag. I can probably manage to run one cable top-floor to bottom (multi-storey house) for a backhaul for Mesh routers, and just rely on a few small switches and/or wifi (6). Not as good as 10G everywhere but for me, good enough and far, far, far easier to do. Realistically., I only even considered running fibre everywhere because we're about to tear out kitchen and two bathrooms so under-floor access would be easy, but there's still a lot of the house where it'd be a major pain in the chuff to do. Also, I might be prepared to have cable-runs running down a trunk on the wall in living areas but "'er 'ndoors" would put her foot down (on my gonads) if I did that in major living areas, so if it ain't channeled into walls (not an option) or buried under floors (major pain to do) I'm not going to get away with it.

[Apex]

[Apex]

Not directly linked but worth a watch

[spacein_vader]

Similar to Saracen my need for 10gb wasn't there. My Internet connection leaves plenty of headroom even on gb ethernet, the only internal networked device that would benefit from the increased bandwidth is my home media server. Having done some research the cheapest way to do that was a cheap Pcie ethernet card added to it and enabling link aggregation.

[Saracen999]

Not directly linked but worth a watch

Interesting watch, but way overkill for my needs. Two £1000 NASs, 16 8TB drives, three XEON NUC's and $200/year for VMWare. Plus M.2 SSDs, RAM upgrades and a couple of UPSs. Hmmm. My level is more like a single NAS and deciding whether to stretch 4-bay to 6-bay, and a USB drive or two for backup. Maybe (and I say maybe because power outages here are about a once/decade thing, or less) a single UPS for the NAS.