Introduction
Update 10/3/16 – Added information about Ubiquiti 3×3
In my previous reviews of eero and Luma, I haven’t done a very good job of testing their key feature: mesh. Mesh is this year’s answer to all that ails your Wi-Fi networks’ performance. Just buy a three pack of cute little devices for somewhere between $300 and $500 and you’ll have the fastest, smoothest running Wi-Fi network ever…honest!
Mesh wireless systems rely on wireless connections among multiple nodes to work their magic. So our normal throughput vs. attenuation chamber testing doesn’t tell the full story. So I took four products, eero, Luma and Ubiquiti Networks’ Amplifi and Amplifi HD (yes, a full review is coming), set them up to force a multi-hop configuration, then measured performance at the root, middle and far-end nodes.
Setup
Mesh wireless makers would love buyers to set their systems up in a perfect configuration. This would be with the root node (the one hardwired to your modem or current router) in the center of your home and the two other nodes equidistant from the root node. And, oh yeah, make sure there is clear line of sight between everything.
This recommendation is pretty silly, given the main reason people need to resort to wireless extenders, BHR (Big Honkin’ Routers) and now wireless mesh systems in the first place is to make up for the fact that they can’t place their router in the center of their home. Of all the vendors trying to sell mesh wireless only NETGEAR acknowledges that most routers live at one end of a home, many times in the basement or lower level—the most convenient place for an installer to run a line. Their new Orbi assumes that the router unit will be stuck against a wall as shown in the promotion image below.
NETGEAR knows how the real world works
Since SmallNetBuilder is all about what happens in the real world, we tested all four products placing the root node in a downstairs office at one end of SNB Manor, the second node in an upstairs allway pretty much at the center of the house and the third node in our beloved kitchen zone of Wi-Fi death. The approximate locations are shown in the floorplan below.
Mesh test locations
The photos below provide a better idea of the node location surroundings.
Test location surroundings
Note the Hallway and Kitchen node locations appear closer together in the floor plan than they actually are. I used the same locations for all products, chosen to accommodate both wall-plugged and desktop formats and to ensure that I could test a true two-hop scenario. In other words, I didn’t want the Kitchen node to be able to see the Office node.
I’ll admit that better locations could have been chosen for each product. But none of the apps told me to move the nodes to improve the signal either during or after setup. In fact, only Amplifi provides any indication of signal strength between nodes or how nodes are connected.
The left screenshot shows Amplifi’s two "mesh points" in single-hop mode, i.e. each directly connected to the router and with backhaul links of pretty much equal signal strength. The right screenshot shows the mesh points in a two-hop configuration, with a weak backhaul link between the router and first mesh point and a much stronger link between the first and second mesh point.
Amplifi shows you how things are connected
I think the right side Amplifi screenshot accurately illustrates the conditions all four products encountered; low signal level from Office to Hallway node and stronger signal level from Hallway to Kitchen node.
Before testing, I had eero and Amplifi products check for firmware updates. eero was running v 1.1.4 and Amplifi ran version 1.3.0. Luma provides neither a way to see firmware version nor a way to check for updates.
Test Method
I tried to use each system as a typical user would, i.e. not mucking with anything in the app other than what was needed to get set up. Basically this meant setting the network name (SSID) and wireless encryption key. In each case, I left other app settings at their defaults and set each system up with a single SSID for both bands. Only Amplifi lets you choose the band to use for backhaul (it defaults to 5 GHz) to both mesh points and whether to steer devices to the 5 GHz band (the default). I left both settings at their defaults. I made no attempt to figure out or control which band was used for backhaul during testing.
The test client was my trusty Lenovo x220i with a NETGEAR A6200 AC1200 class USB adapter plugged in and the notebook’s internal wireless disabled. I checked the adapter’s properties to ensure that both bands were disabled and no band preference was set. Any roaming settings were left at their default "auto" settings. In other words, I tried to have the client be as compliant as possible to any "guidance" the mesh system might provide for roaming or band steering. For each test, the laptop was location 5 – 6 ft. from the node being tested.
I used IxChariot running the throughput.scr test script with test file size set to 5,000,000 bytes and TCP/IP protocol; my usual setting for wireless AC testing. Each test was run once with a single connection for one minute and the average throughput recorded. Windows’ wireless network connection status doesn’t provide the BSSID of the wireless connection, i.e. the MAC address of the AP connected to. So I used NETGEAR’s genie utility and recorded each connection’s BSSID and link rate to confirm that the test client was connected to the proper node.
The Results
In general, the laptop connected to the 5 GHz band for all tests. The only exception was the Amplifi HD. When it roamed to the Hallway node, it initially connected in 5 GHz for the uplink test, which was run first. But before I started the downlink test, I noticed it switched to 2.4 GHz. So I reran the uplink test, then ran the downlink. The client stayed connected in 2.4 GHz for the Kitchen tests, too. But when I forced the client to reconnect, it connected to 5 GHz.
The first graph tells the tale for downlink throughput for each of the three test locations. The Kitchen – Reconnect results reflect a second test run in the Kitchen location, after forcing the client to disconnect, then reconnecting. This was necessary because in all cases, the test client always roamed from the Office to Hallway node, but did not roam from Hallway to Kitchen. To state more clearly, the Kitchen – Reconnect test is the one to look at for two-hop performance. The Hallway test represents best case single-hop performance.
The standard Amplifi’s Office throughput is significantly lower than the Amplifi HD and also eero and Luma. But that is its only weak point. Otherwise, the throughput of the two is evenly matched, with the HD even falling behind the standard Amplifi in the first Kitchen run. The clear winner for downlink is eero. Although it doesn’t turn in the highest throughput in the best-case Office run, it does in all the other tests, including the two-hop Kitchen – Reconnect test. For two-hops, the clear loser is Luma, barely eking out a connection at 0.7 Mbps.
Mesh throughput summary w/ Hallway node – downlink
Uplink shows a more significant difference between Amplifi and Amplifi HD once you get a hop or two in the connection. Luma again does not do well in the two-hop Kitchen – Reconnect test. eero again does well, but bows to the Amplifi HD in the two-hop test.
Mesh throughput summary w/ Hallway node – uplink
Moving A Node
Location matters when placing mesh nodes. However as noted earlier, help for this important step is largely missing from mesh products so far. So I decided to see how much moving the center node could affect performance. The composite / x-ray view of my home’s floorplan shows the single downstairs office node location along with the Hallway and Kitchen office locations. It’s clear the Hallway location isn’t the best.
So I threw WAF concerns to the winds and moved the middle mesh node to the new Living Room location shown. This location puts the center mesh node on a more direct path between the Office and Kitchen nodes. This should provide higher throughput for the Living Room node to pass along to the Kitchen node.
Mesh test locations – composite view w/ Living Room location
The photo shows where each node type (desktop / wall-plugged) was located and its surroundings. No attempt was made to aim any of the nodes to tweak performance.
Living room location surroundings
The downlink comparison again shows eero with strongest one-hop performance (Living Room). Luma takes the award for most improved two-hop performance (Kitchen – Reconnect), improving from a barely-connected 0.7 Mbps to a quite respectable 58 Mbps, second only to eero’s 70 Mbps and ahead of both Amplifis.
Mesh throughput summary w/ Living Room node – downlink
Uplink shows Luma this time taking top honors in the two-hop test with 64 Mbps, up from its previously dismal 1 Mbps. Amplifi HD edges ahead of eero for the top spot in the single hop (Living Room) test with 120 Mbps vs. eero’s 107 Mbps.
Mesh throughput summary w/ Living Room node – uplink
In general, both one and two-hop throughput improved. With the middle node in the Hallway location, only a single product, eero for downlink and Amplifi HD for uplink managed throughput over 40 Mbps. Moving the middle node to the Living Room location boosted two products over 50 Mbps for downlink and three running uplink. A significant improvement!
You may have noticed asterisk on Luma’s Kitchen – Reconnect results, which are the same as the Kitchen results. That’s not a data entry error. Luma apparently decided that the best performance for the client sitting at the Kitchen test location was via the Living Room node. Try as I might, I could not get the test client to connect to the Kitchen node, even after changing client roaming settings. Since I just copied the Kitchen results, Luma’s Kitchen – Reconnect results are the only case where Kitchen – Reconnect represents single-hop results.
So you don’t have to flip back and forth between the two sets of charts, I created charts showing throughput % change between the test runs with the middle mesh node in the Hallway and with it in the Living Room locations. There really shouldn’t be much change for the root node, since in both cases the client connected directly to the base node. So I’ll chalk the Amplifi’s 160% change up to an anomaly.
eero benefits the least from the middle node move, Luma the most and both Amplifi’s somewhere in between. Note I had to limit the chart maximum because of Luma’s extraordinary improvement.
Throughput % change from changing middle node position – downlink
Luma once again gains the most on uplink for both one and two hops. But keep in mind again, Luma’s two hop improvement is a mix of two hop and one hop results. eero gets a significant two-hop boost and not so much for one-hop.
Throughput % change from changing middle node position – uplink
Closing Thoughts
I think it’s pretty clear that eero turned in the best overall performance. No, it didn’t produce the highest downlink throughput with the test client directly connected to the root node. And in the two-hop uplink test it basically tied with the Amplifi with about one-third the throughput of the Amplifi HD. But in the single-hop Hallway test, it had almost 3X the performance of the other three products on downlink and tied the Amplifi HD on uplink.
eero also turned in the best one and two-hop performance when the middle mesh node was moved to the Living room. Although this move greatly improved Luma’s results, eero still did better on downlink. Luma bested eero only on two-hop uplink, but again, Luma would not let me test two-hop in this case.
But the big takeaway is what Wi-Fi pros already know: wireless hops can really eat up throughput. So no matter which mesh wireless system you choose, be prepared to experiment with node locations. Unfortunately, only Amplifi provides signal strength information to guide mesh node placement and also provides a clear indication of how nodes are connected. With the others, you’re on your own to devise your own methods to determine best node placement. Let’s hope vendors improve the situation, because it’s clear mesh node placement matters…a lot!
Update 10/3/16
After this review posted, Ubiquiti contacted us to protest our use of a 2×2 test client. Amplifi HD uses 3×3 AC1750 class both in its router and "mesh point" extender and Amplifi uses 3×3 AC1750 in its router and 3×3 N900 class in its mesh point. So it is true our 2×2 STA will not show the higher throughput that Amplifi should be capable of when used with 3×3 devices.
However, since eero and Luma are 2×2 designs, using a 2×2 test STA provides a level playing field for comparison among all three products. There is also nothing in the use of a 2×2 test STA that should prevent Amplifi from using its 3×3 architecture to obtain higher-bandwidth backhaul between mesh points and router. But our tests do not show a noticeable performance advantage from Amplifi or Amplifi HD.