Introduction
It’s been a long, slow slog for the engineers developing Wi-Fi 6 chipsets and products. I’ve been in the thick of it, trying to develop Wi-Fi 6 test methods that can show whether these draft-standard products deliver on the benefits they promise. That effort has been hampered by the fact that all the consumer Wi-Fi 6 products on the market to date have not had either OFDMA or AX MU-MIMO enabled.
That’s right, despite what you read on their websites and product boxes, Wi-Fi 6 routers generally don’t support key technologies that they advertise. The exceptions to this in the consumer Wi-Fi realm are NETGEAR’s Qualcomm-based RAX120 and ASUS’ Broadcom-based RT-AX88U. The former enabled OFDMA DL (downlink) only for both bands back in August, while ASUS pushed out a Beta firmware in late July that supports OFDMA DL in 5 GHz only.
MU-MIMO support has also generally been MIA for AX clients. When it is enabled, it will be for downlink only; AX MU-MIMO UL (uplink) won’t be supported until a later revision of the 11ax spec.
My takeaway from this is that it’s still too early to post meaningful results trying to determine whether OFDMA delivers on its promises of increasing total bandwidth capacity and reducing latency. So I’m going back to the basics for this article and just testing rate vs. range (throughput vs. attenuation). But unlike my first pass at this last December (First Peek At Wi-Fi 6: ASUS RT-AX88U & NETGEAR RAX80), this time I’m going to look at how AX routers perform using an AX client.
The Products
I tested the five draft AX routers for this go-around, shown in Table 1, three based on Broadcom’s AX platform, one on Qualcomm’s and one Intel based.
Both the ASUS RT-AX88U AX6000 Dual Band 802.11ax WiFi Router and NETGEAR Nighthawk AX8 8-Stream AX6000 WiFi Router are built on essentially the same Broadcom platform, i.e. a Broadcom BCM49408 64 bit quad-core @ 1.8 GHz processor and two BCM43684 4 stream an/ac/ax radios. So the most direct comparison can be made between these two products.
The ASUS ROG Rapture GT-AX11000 is the only tri-band router tested, but its Wi-Fi design is essentially the same as the RT-AX88U’s. I didn’t have NETGEAR’s tri-band AX12 Nighthawk RAX200 on hand, which has essentially the same design.
The NETGEAR Nighthawk AX4 is the lone representitive of Intel’s AX router efforts that I’ve been able to get my hands on. It’s the only two-stream design in this test and the most aggressively priced at around $100 less than the others.
I didn’t open up my samples, but gathered the information below from the soon-to-be departed WikiDevi and around the ‘net.
Updated 9/23/19: Corrected values for RAX80.
| Product [Firmware] | Price | Streams (2.4/5GHz) |
CPU | RAM | Flash | Radios |
|---|---|---|---|---|---|---|
| ASUS RT-AX88U [9.0.0.4.384_6396_BETA] |
$300 | 4/4 | Broadcom BCM49408 64 bit quad-core @ 1.8 GHz |
1024 MB | 256 MB | BCM43684 (x2) 4-stream an/ac/ax radio |
| ASUS GT-AX11000 [3.0.0.4.384_6436] |
$400 | 4/4 | Broadcom BCM49408 64 bit quad-core @ 1.8 GHz |
1024 MB | 256 MB | BCM43684 (x3) |
| NETGEAR RAX40 [V1.0.2.60] |
$192 | 2/2 | Intel/Lantiq GRX350 dual-core @ 800 MHz |
512 MB | 256 MB | Intel WAV654 (x2) |
| NETGEAR RAX80 [V1.0.1.64] |
$300 | 4/4 | Broadcom BCM49408 | 1024 MB | 512 MB | BCM43684 (x2) |
| NETGEAR RAX120 [V1.0.1.90] |
$400 | 4/8 | Qualcomm IPQ8074 quad-core @ 2 GHz |
1024 MB | 512 MB | Qualcomm QCN5054 (5 GHz x2) Qualcomm QCN5024 (2.4 GHz) |
Table 1: Component summary
How We Tested
The goal of this round of tests is to assess the performance seen by a two-stream AX (aka Wi-Fi 6) device by running classic rate vs. range (RvR) tests. All routers were set as follows:
- HE features enabled – This enables AX link rates and 1024 QAM modulation on both bands
- 2.4 GHz: Channel 6, 20 MHz bandwidth. Since most devices operate in crowded 2.4 GHz environments, I kept our traditional setting.
- 5 GHz: Channel 36, 20/40/80 MHz bandwidth
I did not run tests with 160 MHz bandwidth enabled on 5 GHz. Enabling 160 MHz requires using DFS channels, which aren’t available in all countries or reliably where public safety (including airport) radar is in use. I also found using DFS channels can significantly increase 5 GHz association times because devices (STAs) can only passively scan for available APs. So instead of being able to send out a probe request and waiting for APs to reply, a STA using DFS channels must wait until an AP announces itself. You wouldn’t think this would make much difference, but I found when switching from running 2.4 to 5 GHz tests, it took over a minute to find and associate to the router’s 5 GHz SSID.
I did, however, also test the routers with an AC station. An octoScope Pal24 was used for the 2.4 GHz tests, set to use two streams with 20 MHz bandwidth. Since the Pal24 is configured to operate as an N device, this setting will limit the maximum link rate to 144 Mbps. An octoScope Pal5 was used for the 5 GHz tests, set to 80 MHz bandwidth to provide the maximum two-stream 802.11ac link rate of 867 Mbps.
Each router was centered on the turntable in an octoBox 38 test chamber, as shown below. This box is part of the TR-398 testbed and will likely be the one used once the next revision of the SNB wireless test process is finalized. Only the left-hand set of antennas was used in these tests.
ASUS GT-AX11000 in test chamber
I chose Intel’s AX200 M.2 module as the Wi-Fi 6 STA (graciously supplied by Intel). Given it’s the only game in town for laptop/notebook makers who don’t want to embed a Wi-Fi chip directly on their board, the AX200 is likely to become the de facto Wi-Fi 6 laptop client and maybe SmallNetBuilder’s Wi-Fi 6 standard test STA.
Unlike Broadcom’s BCM4375 smartphone combo chip, which is used in Samsung’s Galaxy S10 family and Apple’s iPhone 11, the AX200 supports 160 MHz bandwidth. Although I chose not to test 5 GHz 160 MHz bandwidth performance this time, I want the option for future testing.
The last reason for choosing the AX200 was that it provides the option of using a cabled connection, maximizing signal level so that the maximum Wi-Fi 6 MCS11 can be reached. This equates to maximum two-stream 2.4 GHz link rates of 287 and 574 Mbps for 20 and 40 MHz bandwidths, respectively and 1201 and 2402 Mbps for 80 MHz and 160 MHz bandwidths in 5 GHz. So with our test bandwidths of 20 MHz @ 2.4 GHz and 80 MHz @ 5 GHz, these represent link rate increases of 2X for 2.4 GHz (287 Mbps / 144 Mbps) and 1.38X for 5 GHz (1201 Mbps / 867 Mbps).
Aaeon pico ITX board with Intel AX200
The AX200 is hosted on an Aaeon PICO-APL4-A10-0003 board, shown above. This is a pico-ITX format board with an Intel N4200 Pentium processor, 4 GB of RAM and 256 GB SSD, loaded with both Windows 10 Pro and Linux Mint 19.2. Although I prefer to run Linux to access more radio statistics, I found a significant uplink throughput problem that Intel is in the process of fixing. The AX200 Windows drivers are in much better shape, but you’ll see there is perhaps still some interoperability work to be done. I used the latest 21.30.3.2 driver.
The Rate vs. Range (RvR) test shows how throughput (rate) changes as signal level is varied (range). Instead of using physical distance to vary the signal by walking a laptop around, the router is placed in an RF-tight anechoic chamber (the octoBox) and programmable RF attenuators are used to change the signal. The diagram below shows the test setup.
RvR test setup
For 2.4 GHz, the attenuation was ramped from 18 to 81 dB in 3 dB steps; for 5 GHz, the range was 0 to 54 dB in 3 dB steps. Each test ran five simultaneous connections of unlimited rate TCP/IP traffic for 35 seconds, with the first five seconds discarded from the test average at each attenuation level. The 18 dB of starting attenuation for the 2.4 GHz tests was necessary to avoid overloading the router, Pal or AX200 radios. The router under test was rotated 360 degrees during each test run and the router and STA were disassociated and reassociated between downlink and uplink test runs.
The test sequence was:
- 2.4 GHz @ 20 MHz downlink
- 2.4 GHz @ 20 MHz uplink
- 5 GHz @ 80 MHz downlink
- 5 GHz @ 80 MHz uplink
The published results represent single test runs. When results looked odd, selected runs were repeated, with best obtained results shown.
AC vs. AX Performance – ASUS GT-AX11000
I’ll first show how each router did with AC and AX STAs. First up is the ASUS GT-AX11000, the most expensive router in this round-up.
The 2.4 GHz downlink plot shows the 2X throughput improvement for some of the points with lower attenuation / higher signal, as predicted by the link rate ratio described earlier. The sharper throughput falloff between 39 and 51 dB attenuations decreases the advantage, but it moves back toward 2X as signal level falls.
There are actually two AC STA lines on the curve, the AC200 AC mode line represents the Intel AX200 adapter limited to AC link rates by using the Windows Advanced Wireless Properties selector. The Pal24 and AX200 track pretty closely through the strong-to-mid level attenuations. But as signal levels fall after 54 dB attenuation, the AX200 moves ahead, maintaining better throughput for the rest of the test run.
ASUS GT-AX11000 2.4 GHz – downlink
The 2.4 GHz uplink plot shows an even greater throughput advantage between AX and AC STA, as well as a better-behaved curve. We also once again see the AX200 outperform the Qualcomm-based Pal24 at attenuations beyond 54 dB.
ASUS GT-AX11000 2.4 GHz – uplink
The 5 GHz downlink plot, however, shows a bad, but consistent story for both AC and AX STAs. In all cases, throughput quickly falls off and slowly increases in the mid attenuation range, then once again declines at higher attenuations. This is what I meant when I earlier said that there appears to be some interoperability issues to iron out.
ASUS GT-AX11000 5 GHz – downlink
5 GHz uplink shows even wonkier performance for the stronger signal levels (lower attenuations) for the AX200 in both AX and AC modes. The octoScope Pal5 AC STA, however, looks just fine.
ASUS GT-AX11000 5 GHz – uplink
AC vs. AX Performance – Intel RAX40
NETGEAR’s RAX40 is the lone representative of Intel’s Wi-Fi 6 platform in this round-up. It’s also the only two-stream router and the most affordable at around $200. Let’s see what saving some coin on your first AX router gets you for Wi-Fi 6 performance.
This time, two AX200 runs were done, but instead of switching the AX200 to AC mode, it was left as an AX STA, but using the Linux driver.
The 2.4 GHz downlink plot shows the expected 2X throughput gain (actually better) from the AX200 using the Windows driver. But performance with the Linux driver is significantly lower, running somewhere between the AX200 / Win 10 and octoScope Pal24 plots at low-to-medium attenuation values. Things settle down at the 48 dB attenuation mark and are more predictable after that. The AX200 shows it’s a superior adapter than the Qualcomm-based Pal24 from 57 dB attenuation onward and finally matches the Windows 10 driver from 69 dB on.
NETGEAR RAX40 2.4 GHz – downlink
The 2.4 GHz uplink plot shows why I prefer not to use the AX200 as a Linux STA until Intel sorts out the driver problems. Multiple runs had to be made before getting one that didn’t disconnect very early. Note the adapter didn’t disconnect at 33 dB attenuation; it remained connected, but produced very little throughput.
NETGEAR RAX40 2.4 GHz – uplink
The 5 GHz downlink plot shows an interesting mix of results, with the AX200 on Win 10 turning in the best performance. The AX200 with Linux showed the rapid throughput decrease we saw on the ASUS GT-AX11000, while the octoScope Pal5 had an, uh, interesting run.
NETGEAR RAX40 5 GHz – downlink
5 GHz uplink again shows the AX200 Linux driver has some significant problems. Remember this is with an Intel STA connected to an Intel AP! The AX200 on Windows produced the best run, although it had problems at higher signal levels. The Pal5 also had a throughput dip early in the run but recovered, only to rapidly fall off between 24 and 30 dB attenuation.
NETGEAR RAX40 5 GHz – uplink
AC vs. AX Performance – NETGEAR RAX120
NETGEAR’s RAX120 is also a Wi-Fi 6 lone wolf in this round-up, using Qualcomm’s Wi-Fi 6 platform. It’s also unique in its use of a 5 GHz 8 stream radio that produces only four-stream link rates. The 2.4 GHz side uses a straight four-stream AX architecture.
From here on, I’ll be showing only results using the AX200 w/ Windows driver to measure Wi-Fi 6 performance and Qualcomm-based Pal to show what a two-stream N/AC STA can do.
The 2.4 GHz downlink plot shows that the AX client fails to produce the expected higher throughput. The test script also runs netsh wlan show interface on the STA every few seconds and logs it to a CSV file. That command reports receive and transmit link rates, both of which were the expected 287 Mbps. So throughput should have been up around 200 Mbps, as the same test on other products has shown.
NETGEAR RAX120 2.4 GHz – downlink
The 2.4 GHz uplink plot shows the expected higher AX throughput, with both AC and AX STAs showing generally clean plots.
NETGEAR RAX120 2.4 GHz – uplink
The 5 GHz downlink plot shows the AX STA with lower throughput than the AC STA, even though the AX STA reported the expected 1201 Mbps link rate at the strongest signal levels. The good news is that the AX STA maintains mugh higher throughput than the AC STA as signal levels drop.
NETGEAR RAX120 5 GHz – downlink
5 GHz uplink again shows results similar to downlink; lower throughput at higher signal levels, higher throughput at lower levels. But the Qualcomm-based AC STA has an even greater throughput advantage over the Intel-based AX STA at high to medium signal levels.
NETGEAR RAX120 5 GHz – uplink
AC vs. AX Performance – NETGEAR RAX80
The last two products give Broadcom’s BCM43684 AX radio a chance to show what it can do, starting with NETGEAR’s RAX80.
The 2.4 GHz downlink plot shows the expected throughput advantage that AX link rates bring over AC.
NETGEAR RAX80 2.4 GHz – downlink
The 2.4 GHz uplink plot also shows AX’s throughput advantage, except at the lowest signal levels.
NETGEAR RAX80 2.4 GHz – uplink
The 5 GHz downlink plot shows an RvR curve very similar to the one produced by the ASUS GT-AX11000, which uses the same radio SoC.
NETGEAR RAX80 5 GHz – downlink
Unlike the ASUS, the NETGEAR’s 5 GHz uplink plot is better behaved, but again shows the AC STA having a throughput advantage at strong to medium signal levels
NETGEAR RAX80 5 GHz – uplink
AC vs. AX Performance – ASUS RT-AX88U
Finally, we end as we began, with a Broadcom-based ASUS router. ASUS’ RT-AX88U is basically the same design as the GT-AX11000, but with only one 5 GHz radio and no 2.5 Gbps Ethernet port and around $100 cheaper.
The 2.4 GHz downlink plot is similar to the GT-AX11000’s for the AX STA, but better behaved with the AC STA.
ASUS RT-AX88U 2.4 GHz – downlink
The 2.4 GHz uplink plot shows no surprises.
ASUS RT-AX88U 2.4 GHz – uplink
The 5 GHz downlink plot once again shows an unusual curve, but it’s the same as the other two Broadcom-based routers. It looks like Broadcom has some work to do here.
ASUS RT-AX88U 5 GHz – downlink
The 5 GHz uplink plot is better behaved than the GT-AX11000’s, but the AX curve drops quickly, then flattens out and the AC curve has a roller-coaster shape. Not what you’d want from a $300 router.
ASUS RT-AX88U 5 GHz – uplink
Product Comparison
Now that the AX vs. AC performance for each product has been shown, it’s time to put the products head-to-head.
The first 2.4 GHz downlink plot compares all five products with the Intel AX200 STA. All show higher throughput from the higher AX link rate except the Qualcomm-based NETGEAR RAX120. As noted earlier, the STA reported the expected 287 Mbps link rate at the start of the RAX120 run. I don’t know why throughput didn’t follow the higher link rate.
AX STA Throughput Comparison – 2.4 GHz Downlink
The same conditions, but with the Qualcomm-based Pal24 configured as an AC (actually N) STA shows the Intel-based NETGEAR RAX40 as the outlier. The RAX40’s different performance with low atteuations (higher signal levels) is likely due to poor rate selection.
AC STA Throughput Comparison – 2.4 GHz Downlink
The 2.4 GHz uplink plot with AX STA shows the Intel-based NETGEAR RAX40 having the lowest plot. Since the plot falls off faster than the others as signal levels drop, I suspect we’re seeing the effects of the RAX40’s two-stream radio vs. the four-stream designs of all the other routers.
AX STA Throughput Comparison – 2.4 GHz Uplink
Switching to the AC (N) STA plot doesn’t show the same weakness for the NETGEAR RAX40. All products produce essentially the same results.
AC STA Throughput Comparison – 2.4 GHz Uplink
Moving on to the 5 GHz downlink plot with AX STA, we see the widest range of results. It appears that the Qualcomm-based NETGEAR RAX120 turns in the most normal performance. followed by the Intel-based NETGEAR RAX40. All three Broadcom-based routers do not perform well in this benchmark, but show remarkably similar RvR curves.
AX STA Throughput Comparison – 5 GHz Downlink
The same benchmark using an AC STA again shows the NETGEAR RAX120 as the best of the bunch. The RAX40 doesn’t fare as well this time, with throughput oddities at both start and end of the run.
AC STA Throughput Comparison – 5 GHz Downlink
If I had to declare a winner for 5 GHz uplink with AX STA, it would probably be the NETGEAR RAX120. It had the most well-behaved performance and highest starting throughput. Note that all three Broadcom-based routers did not behave the same in this benchmark, with the most expensive ASUS GT-AX11000 showing the widest variations with strong to medium level signals.
AX STA Throughput Comparison – 5 GHz Uplink
Finally 5 GHz uplink with our AC STA shows the least desirable behavior from the Broadcom-based ASUS RT-AX88U and Intel-based NETGEAR RAX40.
AC STA Throughput Comparison – 5 GHz Uplink
Closing Thoughts
This is a lot of data to absorb, so how about some key takeaways to help digestion?
• In general, 2.4 GHz AX performance was better behaved than 5 GHz. Most products produced around 2X throughput when used with an AX STA vs. an AC STA. Right now, this is likely to be the thing that most buyers would notice from upgrading both router and device to Wi-Fi 6. Note that you’ll get this benefit even from using 20 MHz bandwidth and even with the lowest priced two-stream NETGEAR RAX40.
• There was no throughput gain observed from AX vs. AC when using 5 GHz. Even though our test STA got the expected 1201 Mbps link rate with all routers with the strongest signal, this did not translate to higher throughput. In fact, both AC and AX STAs were only able to get 700 Mbps at most and usually below that. 5 GHz uplink with the AC STA actually produced the highest throughput!
• 5 GHz downlink performance with both AX and AC clients needs work. Only NETGEAR’s Qualcomm-based RAX120 and Intel-based RAX40 did not produce anything remotely resembling a normal RvR curve. The RAX120 was best behaved with both AX and AC STA running downlink and uplink. 5 GHz uplink was generally better for both AX and AC STAs for Broadcom-based routers, but was not without problems.
Even after almost a year after the first products started shipping, Wi-Fi 6 remains very much a work in progress. Key features (OFDMA, AX MU-MIMO, Target Wake Time (TWT)) are still not enabled by most products and, if these tests are any indication, you’re unlikely to see performance gains in from 5 GHz.
If you have AX devices and feel you must upgrade right now, spend as little as possible. Until OFDMA is enabled and we see whether that helps bring more bandwidth to busy WLANs and reduce latency, there’s no sense in making a large investment that may not pay off.