Our new wired and wireless test processes will include tougher tests and more of them.
Regular readers may have noticed it’s been pretty quiet here lately. Part of this is the recent move of all news coverage over to SNBForums. But the main reason is I’m focusing on birthing new router test processes.
It has been just over six years (!) since we last revised our wired router test process. Since then, many routers now routinely max out our simple throughput tests. So it’s time to work these products a lot harder, to better help guide spending your hard-earned money.
The new process does away with the pair of computers and IxChariot, instead using QA Cafe’s NTA1000, and flagship CDRouter router test solution.
QA Cafe NTA1000
CDRouter is a Linux-based router test suite that includes over 4000 functional and performance tests. I’m working on putting together a test suite that can execute as many of those tests as possible, using a single router configuration. We’ll be checking DNS, DHCP, UPnP, NAT, firewall and many other routing functions on every router we review. As far as I know, this will make SmallNetBuilder the first publication to perform exhaustive and consistent functional router testing.
The NTA1000 will also take over performance testing. We’ll be adding UDP throughput to the test suite and reporting TCP retries as well as UDP % packet loss along with the usual throughput results.
One thing we won’t be testing, however, is IPv6. The reason is that implementations still vary widely and don’t all support autoconfiguration or even a common set of IPv6 DHCP connection types. I found I was spending too much time just trying to get a basic IPv6 connection working. So, at least for this round, IPv6 testing won’t make the cut.
On the wireless side, the big news is we’re retiring our trusty NETGEAR R7000 AC1900 test client. Although the exact new standard STA hasn’t been selected yet, I can say it will be a 2×2 device. The reason is to more accurately represent the lower throughput users see from actual devices, most of which aren’t 3×3. The basic wireless test process will remain the same, rotating the router during test in an octoScope RF test chamber and using programmable attenuators to change signal level.
Going in the other direction, maximum wireless throughput testing will be done by the same Veriwave system that is handling MU-MIMO test duties. The Veriwave can emulate up to 4×4 802.11ac STAs. So we’ll be able to test maximum wireless throughput of 4×4 based routers without having to use a matching router configured in wireless bridge mode.
Wireless load testing is also in the works, using the Veriwave system to hammer on wireless routers and APs with a mix of simulated STAs and traffic types. Today’s routers are handing lots more devices, many of which are streaming video while others are gaming, chatting or even just checking email and web browsing.
Of course, all these changes mean the Charts and Rankers will once again reset and require another round of retesting. So it’s going to be a busy spring and even busier summer!