Are people interested in multi-client stress tests on routers?
Are people interested in multi-client stress tests on routers?
I asked this same question on Reddit and I got zero engagement, so perhaps Lemmy has people that care more about their hardware.
I recently decided to use some of the tools provided by Mr Salter (netburn) and I have to ask the community if you want to see multi-client stress tests (4K streaming, VoIP, web browsing) used on a wireless router or if the single-client iperf tests are good enough. Bear in mind that pretty much all publications that still test their devices (most don't) rely on the single-client test method.
For a consumer/household: This is almost entirely unnecessary. Basically any halfway competent name brand router/firewall will have no problems with this. You are more likely to see issues coming from your wifi network (which is probably part of your "router"), but that is also an incredibly situational one depending on your environment (how many neighbors, etc). But LAN->WAN NAT is a "solved problem" as it were and you mostly just want to stress test that speed wise.
For enterprise/hotels? Yeah, that is when you are going to have issues with too many clients. And the answer to that is almost always "buy enterprise hardware" rather than "figure out which netgear router I can tape to the ceiling"
More data is always fun (unless you are the one collecting it...) but I just don't see much benefit from this. And most of the suggestions in this thread are really just ISP tests.
Oh, I agree wholeheartedly that collecting the data is not that much fun, especially since yes, I will have to do it. But I think users may benefit to see if the non-enterprise wireless routers can accomplish a certain task. For example, can that expensive Netgear router actually handle four client devices streaming 4K at the same time? What if we add browsing in the mix? The point of this thread was to get an idea if it's actually worth running these tests (which take quite a bit) and if people are interested in seeing this type of data on the web.
For example, can that expensive Netgear router actually handle four client devices streaming 4K at the same time
Can your ISP? If so, yes. Because ~25 Mbps * 4 is not a lot of data. And the NAT for four clients mapped to the same firewall/router is pretty trivial. And no, adding "browsing" is not going to be an issue.
Again, NAT is easy. And it happens on every single packet (big ol' asterisk on this, but not the venue to get into the specifics), regardless of whether it is one client or two. So what matters is the amount of packets per second that can be processed which these speed tests already cover (albeit, somewhat obfuscated because most people don't understand the network layers).
And in the enterprise case? That is mostly about whether you can run a mesh network, what signal coverage you have, and the total number of clients (and packets) that need to be processed per second. Which... you are either a complete sicko who wouldn't be watching reviews online or you are just going to buy a ubiquiti or omada setup.
Multi client test seems better honestly. I end up running 3-4 iperfs from different clients to a wired server to see how the bandwidth chokes. I wonder how it will be if one of the clients are running the iperf server as well.
Real life workloads like 4k and VoIP with multiple clients seem much more realistic and representative.
Seriously Lemmy is the best. Few minutes in and people are already answering questions. The concept behind the multi-client tests are to SSH into the server and then simulate whatever type of traffic one wants. I have already done it on a couple of wireless routers and it worked great. I hope at least. Iperf can't really accomplish this, as far as I know, it's only one instance on a single client at a time. Netburn seems to be the best tool so far, while keeping things free and open-source.
Seconding your opinion about lemmy :) Do you think you could write up that you did? I would be interested in reading. Found this article on ars as well:
https://arstechnica.com/gadgets/2020/01/how-ars-tests-wi-fi-gear-and-you-can-too/
I'll be checking out netburn.
I totally be interested in this sort of testing methodology being published. Maybe in a wiki?
Getting comparable numbers for buffer bloat and queuing would be great for commercial routers. Of course you would want to compare against Enterprise solution so that people know where on the spectrum they're landing.
Full disclosure I roll my own GLI net open WRT router and I enforce different queues for qos seperation.. i.e. downloading and streaming shouldn't interfere with VoIP calls and gaming
I recently ran into an issue with my home network where I suspected that the current wifi router (3-point mesh) couldn't handle all the clients simultaneously. Not in a manner of throughput, but just with keeping all the devices online in the first place.
I have at minimum 30 clients online all the time, up to a max of 40 or something, depending on who's home or what is active. (Went a bit overboard with home automation and stuff.)
I was getting random disconnects or stalling wifi on some of the devices. The coverage was fine, so I figured it was just the number of wifi signals that was overwhelming the AP.
Point of the story: I was disappointed that absolutely no review/benchmark ever pays attention to this kind of upper limit that seems to exist in practice. It's all range and speed, but never about the maximum number of active clients.
I've replaced the mesh network and everything is fine now. But I had to trial and error that shit...
It isn't really about the maximum number of client devices, it's what they do, what type of standard are they a part of, how far away, the interference (!). This is why it's pretty much impossible to put a number and say: hey, this TP-Link router will handle 30 client devices, while this Asus router goes up to 100.. In a sense, a multi-client stress test kind of addresses this issue, but it kind of doesn't. It's because it's extremely dependent on the conditions that the tester has in their lab/office/home.
One thing to check on a review could be the attenuation as a better factor than the distance (this way, you can reproduce the result in your house even if just with a single-client test).
Not really, to be honest. I'd rather see how compatible a particular router is against popular open-source firmware, how frequently the updates are delivered, etc.
For instance, the Asuswrt Merlin is a pretty good firmware for ASUS routers, but the updates (stable) are irregular - the last stable update currently was two months ago, which to me is unacceptable considering there have been critical vulnerabilities in ASUS routers. Given how malware and botnets are increasingly targeting routers these days, it's imperative that updates get delivered at least once a month - with an out-of-band policy for critical vulnerabilities.
Sounds good to me
One client downloading a torrent, second client viewing 1080p60 video from YT and clients 3&4 transferring a few TB of data through lan.
I would love to see this via WiFi.
When I run iPerf tests I almost always use multiple clients because of exactly what another comment said, it's more realistic and a better representation.
So you basically spawn multiple instances of iperf3 and then connect all clients to a single server (using the same port?) What do you think about checking the latency experienced at the client level when various tests are running at the same time?
Yeah, I spawn multiple instance of iperf3. Checking the latency would be a very useful metric, it should give a good result of what the connection will be like under load.