Why You Don’t Need MU-MIMO

(With apologies to Qualcomm)

Back Story

Regular readers know I call MU-MIMO "the 3DTV of Wi-Fi" because it provides no practical benefit for most consumers. Yet manufacturers insist on churning out more products with it, including MU-MIMO even in products like 2×2 routers and extenders, where it has no chance of providing benefit. The sidebar below explains why.

The consumer Wi-Fi industry has made many missteps in rolling out MU-MIMO. First, there was the initial launch of Quantenna-based MU-MIMO "ready" AC2350/AC2400 class routers back in summer 2014. These include ASUS’ first-to-market RT-AC87U and NETGEAR’s R7500 Nighthawk X4. Some of these still await firmware upgrades to enable MU-MIMO.

AC2600 class routers with Qualcomm’s 4×4 radio were next to market in mid-2015. Routers like Linksys’ EA8500 and NETGEAR’s R7800 Nighthawk X4s actually had working MU-MIMO when they shipped. But as we found in our test of six MU-MIMO enabled routers a little over a year ago, there was a big difference in total throughput gain, which is the whole point of MU-MIMO.

MU, SU Throughput difference – average

When routers using Broadcom‘s 4×4 radio started appearing in late 2015, MU-MIMO took another step backward. Broadcom’s said its BCM4366 4×4 radio supported MU-MIMO. But it turned out the device didn’t have the feature fully baked yet. So while manufacturers still slapped MU-MIMO prominently on the front of product boxes, they also buried disclaimers saying firmware upgrades were needed to turn on MU-MIMO.

To distract buyers from the absence of MU-MIMO, router makers chose to ship the new Broadcom radio in "tri-band" designs, creating gargantuan, expensive, eight-antenna AC5300/AC5400 class routers like NETGEAR’s R8500 Nighthawk X8 and ASUS’ RT-AC5300. The hope was that folks would get some benefit from the second 5 GHz radio and forget about the non-functional MU-MIMO.

Even when lower cost Broadcom-based dual-band AC3100/AC3150 class routers like ASUS’ RT-AC88U and RT-AC3100 started shipping a few months later, working MU-MIMO firmware was still MIA. And when we finally tested a Broadcom MU-MIMO router with allegedly working MU-MIMO firmware in the form of TP-Link’s Archer C3150 in August 2016, we found MU-MIMO performance so poor, that it was essentially non-functional.

It turns out that Broadcom’s problems were not all in firmware, however. I learned late last year that some manufacturers have quietly been phasing a BCM4366 C0 version of Broadcom’s 4×4 radio SoC into production without any visible model number or version change to products. This new version is supposedly required to produce truly functional MU-MIMO in Broadcom-based routers.

Other manufacturers like TP-Link have created a new model version like the Archer C3150 V2 and even created a new FCC ID. It’s interesting that TP-Link’s Class II Change letter refers to the previous BCM4366 B1 as "SU-MIMO".

MU, SU Throughput difference – average

The takeaway here is that even if you load new firmware onto your AC3100/3150/5300/5400 router, it’s unlikely you’ll get decent MU-MIMO performance.

The other problem working against MU-MIMO as a useful Wi-Fi technology is that devices support it have been slow to come to market, particularly in the U.S. The first mainstream U.S. phones to support MU-MIMO were Google Nexus and Samsung’s Galaxy S6.

Both Google and Samsung continue to include MU-MIMO in their phones including 2×2 MU-MIMO in Google’s Pixel and Samsung’s Galaxy S7/S7 Edge. But if you’re an iPhone fan, Apple has yet to support MU-MIMO, and perhaps never will, at least until they settle their kerfuffle with Qualcomm. Tablets supporting MU-MIMO also still aren’t that common.

Today’s MU-MIMO

So the router end of MU-MIMO is a hot mess and support on the device end is nothing to write home about. But it doesn’t end there. It turns out Broadcom also screwed up on the device side of MU-MIMO. Another little secret widely known by industry insiders, but by few consumers, is that Broadcom’s 2×2 MU-MIMO client-side devices fall back to 1×1 mode when connected to a MU-MIMO enabled router. This happens with both Qualcomm and Broadcom-based routers; the problem is on the client side.

I was able to get my hands on a pair of Google Pixels and a pair of Samsung Galaxy S7s to test this. By the way, it’s ridiculously difficult to find out the actual Wi-Fi chip used in smartphones because their Wi-Fi usually comes in a third-party module. For example iFixit’s Galaxy S7 teardown calls out a Samsung 1316S7 Wi-Fi module, while Chipworks’ Galaxy S7 Edge teardown identifies the Wi-Fi module as a Murata KM5D18098 that handles "5G Wi-Fi + Bluetooth". But I’m told the Pixel has Qualcomm MU-MIMO Wi-Fi buried inside and the Samsung S7 has Broadcom.

I set up a little experiment using two MU-MIMO enabled 4×4 routers, a Qualcomm based NETGEAR R7800 (V1.0.2.32 firmware) and Broadcom-based ASUS RT-AC88U (3.0.0.4_382_15850-g7402e99 firmware). The testing was done in open air with phones on shelf at opposite ends of room approximately 8′ away from the router.

Because I no longer have access to ixChariot, I used octoScope’s multiperf3 for testing, running four simultaneous connections with a 2048 kB TCP/IP window size. multiperf3 is a variant of iperf3 that can run simultaneous tests to multiple iperf3 servers.

The test was simple:

1. Disable MU-MIMO on the router
2. Associate the phones and note the device link rates
3. Run multiperf3 test to measure throughput
4. Enable MU-MIMO on the router
5. Repeat Steps 2 and 3

The first screenshot shows the phones associated with the R7800 that had MU-MIMO disabled. Both show link rates indicating a two-stream, i.e. 2×2, connection.

Google Pixel, Samsung S7 Link Rates w/ NETGEAR R7800 – MU-MIMO off

When MU-MIMO was enabled on the router, the Pixel’s link rate stayed the same, but the Samsung’s dropped to the 433 Mbps indicative of a single-stream 802.11ac connection.

Google Pixel, Samsung S7 Link Rates w/ NETGEAR R7800 – MU-MIMO on

Given the 50% reduction in link rate, we’d expect the S7’s throughput to be cut in half, right? Here are two S7’s connected to the R7800 with MU-MIMO off…

Samsung S7 w/ NETGEAR R7800 – MU-MIMO off throughput

…and with MU-MIMO enabled. Average total throughput drops from 624 Mbps to 513 Mbps, a loss of about 18%. Not quite the 50% expected from the link rate drop.

Samsung S7 w/ NETGEAR R7800 – MU-MIMO on throughput

Now let’s see how the Pixels do. MU-MIMO disabled first…

Google Pixel w/ NETGEAR R7800 – MU-MIMO off throughput

… and enabled below. With MU disabled on the R7800, total average throughput is around 616 Mbps and with it enabled, it’s around 760 Mbps, a gain of about 23%. This is far from the "up to 3x faster" Qualcomm claims in its MU-MIMO promotional material, but at least still a gain.

Google Pixel w/ NETGEAR R7800 – MU-MIMO on throughput

1 Stream vs. 2 Stream

When we were still testing MU-MIMO performance, we used Veriwave test gear that emulated multiple single-stream (1×1) MU-MIMO devices (STAs). But newer phones are using two-stream radios, which some routers may not handle optimally.

For example, Qualcomm’s older 4×4 QCA9980 radio can provide peak total throughput for three 1 stream or one two stream and one 1 stream devices. If presented with two two-stream MU-MIMO devices, it’s likely the second device gets treated as SU, with no resulting MU-MIMO throughput gain.

According to the slide below taken from a June 2015 Qualcomm presentation, the QCA9984, used in newer routers like NETGEAR’s R7800 and Synology’s RT2600ac, can support two, two-stream devices.

Older MU-MIMO devices support only three devices per frame
(Image credit: Sogi.com.tw)

So let’s compare MU-MIMO throughput gain using two 1×1 and two 2×2 devices. I used two octoScope Pal-5 partner devices, which use the QCA9984 and allow setting the number of MIMO streams.

I ran tests with two Pal-5’s connected to the NETGEAR R7800, which also uses the QCA9984 in its 5 GHz radio and then the ASUS RT-AC88U, which uses a Broadcom BCM4366. I unfortunately did not note the revision level of the device when I opened it up, so don’t know if it’s using the C0 rev allegedly required for proper MU-MIMO support. As the data will show, I suspect it’s not.

Each router being tested was placed in the center of the octoScope 38 chamber. One Pal-5 connected to antennas on the right side of the chamber; the other connected to antennas on the left side. Programmable attenuators were used to adjust signal levels reported by the Pals to be approximately equal. The same traffic used for the previous phone testing was used, i.e. multiperf3 running four simultaneous connections with a 2048 kB TCP/IP window size.

The bar charts below compare total throughput from the two STAs, since that’s what MU-MIMO is supposed to increase. The pairing of Qualcomm-based STA and router using the same QCA9984 chipset should give MU-MIMO its best shot. The results show a 62% gain in total throughput using the pair of 1×1 STAs, but only 17% gain using 2×2’s.

1 vs. 2 stream MU-MIMO Gain – NETGEAR R7800

As unimpressive as that may be, pairing a Broadcom-based router and Qualcomm-based STA is even worse. The pair of single-stream STAs manage to get a 42% total throughput gain. But the pair of 2×2 STAs results in a 58% total throughput loss!.

1 vs. 2 stream MU-MIMO Gain – ASUS RT-AC88U

Here’s what the ASUS throughput plots looked like, with MU-MIMO off…

ASUS RT-AC88U – 2 stream Pal STA – MU-MIMO off

…and with MU-MIMO enabled.

ASUS RT-AC88U – 2 stream Pal STA – MU-MIMO on

Useless MU-MIMO

So it’s bad enough that a buggy feature gets prominently hawked as a product advantage when it has a chance of providing benefit. But when it has no chance of doing anything useful, it’s way past time to call it out.

Such is the case with MU-MIMO in two-stream products. I keep seeing MU-MIMO listed as in the specs of Wi-Fi extenders and multi-node Wi-Fi systems like NETGEAR’s Orbi that have 2×2 client-facing radios.

If we use the old number of streams -1 rule of thumb for the number of single-stream devices that provide peak MU-MIMO gain, we get 1, which makes no sense. So let’s say a two-stream MU-MIMO AP/router will provide peak throughput gain with two single-stream devices.

To test this, I put an Orbi RBK30 into the test chamber, fired up the two octoScope Pal-5’s again and got the results below. Like all Wi-Fi Systems to date, Orbi is based on Qualcomm silicon. So we know it should be capable of supporting proper MU-MIMO operation. Although the original Orbi has a dedicated 5 GHz 4×4 backhaul radio, the client-facing radios in all models are only 2×2.

The plot below shows clearly that MU-MIMO isn’t contributing anything useful here. But at least it’s not decreasing throughput like the Broadcom-based RT-AC88U.

1 vs. 2 stream MU-MIMO Gain – NETGEAR RBK30 Orbi

Closing Thoughts

So there you have it. This is why I don’t waste time testing MU-MIMO anymore. For most cases, it provides no significant benefit. And in other cases, it can result in degraded Wi-Fi throughput.

If you’re one of the few who have a MU-MIMO router and devices that are boosting total throughput in your network, congratulations! But most of us are not that person. At least, like 3D in TVs, MU-MIMO isn’t costing us extra anymore. But it’s certainly not a reason to prefer one product over another.

Discuss this in the Forums