Wi-Fi Roaming Secrets Revealed – Part 4

Introduction

Updated 8/24/18: Added explanation for lack of band steering

It’s been a slow summer for Wi-Fi product introductions. So before the draft 11ax router dam breaks in the fall, I’ve been spending a lot of time thinking about how to put together the next Wi-Fi testbed and exploring new test methods. A lot of that time recently has been focused on Wi-Fi roaming.

The Wi-Fi Roaming Secrets series was a good start at understanding what makes Wi-Fi roaming tick. But it didn’t explore how a client device supporting all three roaming enhancement standards, 802.11k, v and r, behaved, because I couldn’t find such a device.

The studies were also hindered by the time it took to manually analyze device telemetry logs and Wireshark capture files and construct roaming behavior plots like the one below.

This roaming plot takes a lot of time to put together

Test Improvements

Fortunately, two significant developments have significantly improved my ability to assess Wi-Fi roaming behavior. The first is the development of a web-based roaming analysis tool by my test partner, octoScope. The tool takes any uploaded pcap file, automatically analyzes it for key roaming events and generates an RSSI vs. time plot along with a table of detected roaming events.

octoScope Wi-Fi roaming analysis tool

The example plot shown above also places markers (blue dots) corresponding to each detected roaming event on the plot. In the actual tool, mousing over each plotted point—RSSI or roaming event—pops up relevant information. The screenshot above shows the "tooltip" information for one of the RSSI points. The color of the RSSI plot indicates the band (green for 2.4 GHz, red for 5 GHz) for easier identification of band steering events.

Since the plot also extracts RSSI information from the captures, I no longer need to get that information from the STA. So there is no need to run a client-side app to gather the data for readback. Traffic can still be run while testing using either ping or iperf3, since iperf3 apps are available for Windows, iOS, MacOS and Android and ping is built into all modern OSes.

The other roaming test improvement is that I’ve found a device, actually a bunch of them, that support all three roaming assistance standards. The key to this discovery was understanding the fact that many devices (STA) don’t advertise 11r support unless they talk to an access point (AP) that also supports it.

The mesh consumer systems I’ve tested until recently don’t support 11r because it can cause connection problems with older clients. But TP-Link decided 11r support was important and it can be optionally enabled in the second-generation Deco M9 Plus system I recently reviewed.

Once I enabled 11r in the Deco M9 Plus (and confirmed it was at least advertised as supported), I retested my stable of clients and came up with the following results shown in the table below.

Table 1: Device 802.11k/v/r support

iOS devices appear to be the winner here. I was even surprised to see 11r supported in the older iPod Touch G5 and iOS 9.3.5. But the Intel AC 8260 still apparently doesn’t support 802.11r, contrary to what Intel and Microsoft say.

Here’s the Wireshark filter used to look for Authentication, Association and Reassociation frames with RSN Auth Key Management (AKM) Type 4 set…

(wlan.fc.type_subtype == 0x0000 || wlan.fc.type_subtype == 0x000b || wlan.fc.type_subtype == 0x0002)&& wlan.rsn.akms.type == 4

…and the Wireshark capture showing the STA advertising 11r support.

The telltale 11r STA frame

Now that I have an 11k/v/r capable device, I configured the testbed to use it instead of the octoScope Pal, which supports only 802.11v BSS Transition Management Requests. I chose the more recent iPodTouch Gen 6 as the test client. Here’s what it looks like set up in the large octoScope test chamber. Please excuse the elegant test stand. Two octoScope high gain antennas are used to couple the STA into the testbed.

Roaming test setup with device STA

Here’s the block diagram of the setup. The ipod Touch sits in parallel with the Pal-24 and Pal-5 that are used for frame capture.

Roaming test setup with device STA

The Products

I selected the following Wi-Fi systems for roaming test. Table 2 summarizes the 11kvr support specified by the manufacturers. As described in Part 2, each of these standards has many methods within it, with each method affecting roaming differently. So you don’t really know unless you look at the packets what a manufacturer really means when it specs support for 11k,v or r.

Table 2: DUT 11kvr support – manufacturer spec
* = default disabled

Since I did look at the packets for each product with a device supporting all three standards—an iPod Touch – 6th Generation—I was able to construct Table 3 of observed behavior for the 11kvr functions shown.

Table 3: DUT 11kvr support – observed

I was most surprised about the total lack of 11v BSS Transition Management requests in this round of testing, especially because all products tested advertise support for 11v. I found in Part 1 that NETGEAR’s Orbi reliably used these requests to try to band steer the octoScope Pal STA. This time around, even though these requests can be solicited by a STA or sent unsolicited by an AP, none were observed in any of the roams using the iPod Touch client.

The Test

I ran the same roaming test on the five mesh Wi-Fi systems. Here’s the test sequence:

1. Testbed is initialized
2. Packet capture is started on 2.4 and 5 GHz channels
3. Attenuators are set to initial value
4. STA is associated to AP1
5. Continuous ping is started and logged to a file
6. “To” roam done, cross-ramping attenuators from 0 to 63 dB, 1 dB steps, 1 second step dwell time
7. 10 second post-roam pause
8. “From” roam done, same parameters as "To" roam
9. 10 second post roam pause
10. File capture stopped, saved and merged into one file and roam analysis performed

I ran each test at least three times using the iPod Touch G6 and at least once using an octoScope Pal-245 as STA. I used the Pal as a sanity check, since I know its roaming behavior. Although it doesn’t support 11k or r, it does support 11v and reliably responds to 11v BSS Transition Management Requests, at least from NETGEAR Orbi.

The Results

For those who want to cut to the chase, here’s a summary of roaming behavior.

Table 4: Results summary

A few table categories need some explanation:

• Roams: Number of successful roams. Should be 2, since we are testing a round-trip roam
• Band changes: Number of times STA changes bands. This includes non-roaming band-steering events.
• Missed pings: This is the maximum number of sequential ping misses. It provides a rough indication of roaming time. Since ping frequency is 1/second we’re not looking for sub-second roaming behavior. Note the missed pings metric represents pings missed during the return roam only.
• Deauths / Disassocs: Number of deauthorizations or disassociations sent by the AP, not the STA. These are typically used to "encourage" a STA to roam.

In the sections that follow, I’ll show a the result used to create the above summary, along with commentary. You’ll be able to download a PDF of the roaming summary and the pcap file itself.

If you copy this Wireshark dfilter_buttons file into your Windows Users > [username] > Appdata > Roaming > Wireshark folder, you’ll be able to use the filters I’ve created to look for roaming events. Be sure to rename the existing file before copying to save any filter buttons you may have created.

ASUS Lyra Trio

Roaming summary

[Download summary] [Download pcap]

The Lyra Trio set the radios on both nodes for Channel 6 (20 MHz B/W) and 149 (80 MHz B/W). So either could be used for backhaul. We see only the 2.4 GHz radios used in the roam and no sign of band steering attempts.

The roam plots use green to indicate 2.4 GHz and red for 5 GHz, but don’t differentiate channels or BSSIDs (that’s on the list for plot improvements).

Google WiFi roam

The Trio made no attempt to band steer the Apple STA to 5 GHz. If the one disassociation at the start of the run was meant to encourage the STA to move, it was ineffective because the STA ended up connecting to that BSSID.

The roaming summary above shows three sets of 11k neighbor report requests and responses at around 2, 57 and 146 seconds into the run (denoted as Type: Action frames, Category: Radio Measurement on the summary). In each case, however, the report includes only the 2.4 GHz BSSID on the non-associated mesh node. It’s also not clear how or if they affect roaming because they come right after an association/reassociation event. No 11k radio measurement requests or 11v BSS Transition Management Requests are observed in any of the ASUS Lyra Trio captures.

Three pings were missed during one of the roams, putting roaming time at around 3 seconds.

Google WiFi

Roaming summary

[Download summary] [Download pcap]

Google WiFi decided to use 5 GHz Channel 149 (80 MHz B/W) for backhaul and set the root node to 2.4 GHz Channel 6 and the Hop 1 node to Channel 1. Both 2.4 GHz channels were 20 MHz wide. While this means using 2.4 GHz for backhaul is not an option, it does provide more effective bandwidth for devices by giving each node its own channel. I had to bring a third capture device online and tune it to the other 2.4 GHz channel to properly capture everything.

Google WiFi roam

The roam is once again 2.4 GHz only. No deauths or disassociations were observed. The roaming summary above shows that 11kvr doesn’t really affect roams for Google WiFi. Most of the roaming events shown are authentications and association and resassociation requests and responses. The four measurement reports requests Google WiFi makes were not answered by the STA.

Results – more

Linksys Velop Dual Band

Roaming summary

[Download summary] [Download pcap]

Like Google WiFi, Linksys’ Velop Dual Band used a different 2.4 GHz channel on each node. Channel 36 (80 MHz B/W) was used for 5 GHz and Channels 1 and 4 (20 MHz B/W) were used for 2.4 GHz. So backhaul was only possible on 5 GHz.

Linksys Velop Dual Band roam

The Velop roam generated the next to most events, 46 in all. The iPod Touch roamed four times, with one of the roams being from 2.4 to 5 GHz, albeit briefly. While it appears that the STA tried to use Neighbor Reports to guide its roaming decisions, it could not have. Each of four requests was answered by the Velop, but with no information, as shown in the Wireshark detail below.

Linksys Velop Dual Band blank neighbor reports

Velop’s favorite roaming influence tool appears to be disconnecting the STA via disassociations, which it did 11 times. But these attempts appeared to be mostly ineffective. In most cases, the iPod connected right back to the BSSID that disassociated it. Maximum missed pings were three, which puts effective roaming time again around 3 seconds.

NETGEAR Orbi RBK50

Roaming summary

[Download summary] [Download pcap]

Orbi has a dedicated backhaul radio, so both fronthaul bands have no competition for traffic. Orbi chose Channel 8 (40 MHz B/W) and Channel 48 (80 MHz B/W) on both router and satellite mesh nodes.

NETGEAR Orbi RBK50 roam

The roam plot above shows Orbi making the most direct efforts to affect roaming of all products tested, using deauths as its go-to method. The cluster of blue dots around 80 seconds represents a series of 8 (!) deauths trying to disconnect the STA, with four sent in one cluster and another four 1.4 seconds later. After the second volley, the STA sends four deauths back at the unfriendly radio in response, then immediately associates to the 2.4 GHz BSSID on the other AP.

Immediately after associating, the STA asks its new AP for a neighbor report, which Orbi immediately provides. But it includes only the 2.4 GHz BSSID on the other AP it just left.

The Apple STA roamed pretty quickly, missing only one ping.

TP-Link Deco M9 Plus

Roaming summary

[Download summary] [Download pcap]

TP-Link’s latest mesh system shares its radios between front and backhaul. It chose Channel 4 (40 MHz B/W) for 2.4 GHz and Channel 44 (80 MHz B/W) for 5 GHz. Once again, however, the roams stayed on 2.4 GHz.

TP-Link Deco M9 Plus roam

The new Deco takes the prize for most roaming events at 136 (!), almost three times as many as the next most active, Linksys Velop DB. The Deco also made the heaviest use of 11k Radio measurement reports, which account for most of the blue dots you see strewn along the plot above.

The ipodTouch was pretty good about supplying the reports, but sometimes declined the request. When it did respond, it often had to retry its response almost every time, sometimes up to 6 times in a row.

The iPod also provided measurement reports for both bands, but, oddly, for Channel 6 and 42, as shown in the capture detail below.

iPod Touch Radio Measurement Report

There were also two 11k neighbor report requests and responses around 73 and 171 seconds, both made right after associations. All this activity didn’t help roam times. At nine missed pings, the Deco was the worst of the tested group.

Closing Thoughts

What surprised me most from this test was the complete lack of attempts to band-steer the STA to 5 GHz, or at least effective attempts. Only one of the five products got the iPodTouch associated to 5 GHz at any point in the roam. The Linksys Velop DB and NETGEAR Orbi were the only systems to repeatedly use STA disassociation and deauthentication, with the Orbi being more effective, mainly by using deauths and being very insistent.

Update 8/24/18 – Dennis Bland of dB Performance pointed out that the iPod Touch advertises support for only 2.4 GHz channels in its association and reassociation requests. This could be why all the Wi-Fi systems tested did not attempt to band steer it even though it did probe on both bands.

I also saw no obvious attempts before the first association to get the STA connected to 5 GHz. In all but one case (the TP-Link Deco M9 Plus), the iPod Touch was allowed to associate to the first AP 2.4 GHz radio before it even issed 5 GHz probes! So none of the tested mesh systems appear to use delaying probe responses as a band-steering method.

The other band-steering tool that was used effectively by Orbi in Part 1 was 11v BSS Transition Management requests, which everything in this test supports. But nary a BSS Transition Management request was seen; even when I ran a check with the octoScope Pal against each product. It’s possible that I’m misinterpreting the meaning of the BSS Transition bit being set. But for now, this is a puzzling outcome.

If there is a winner to be declared on the basis of roam time, it would be NETGEAR Orbi, which missed only one ping. By the same metric, the biggest loser is TP-Link’s Deco M9 Plus, which lost 9 pings in the roaming process, despite the heaviest use of 802.11k to aid in roaming.

The last takeaway is that I wouldn’t really worry about 802.11r support in either your AP or devices for now. Judging from the inability of any of these systems to support continuous pings during roams, sub-second roams are the least of your worries.

Discuss this in the Forums