Introduction
Judging from some of the threads I see in networking discussion forums, alternative router firmware has its share of fanboys. You frequently see responses that boil down to "…the factory firmware is crap. Just load [insert favorite alternative firmware here] onto your router and everything will be fixed…"
Well, I finally got around to loading up a few of these magic beans and did some basic testing to see what effect they had on routing throughput, variation and session handling. I didn’t really know what to expect, but what I found might give some potential firmware switchers pause before pulling the trigger.
The Firmware
To keep things simple, I chose two of the most popular alternative firmwares for consumer routers: DD-WRT and Tomato.
DD-WRT has a long history. To quote from the DD-WRT Wiki:
The first versions of DD-WRT were based on the Alchemy Firmware from Sveasoft Inc, which is in turn based on the original GPL’d Linksys firmware and a number of other open source projects. DD-WRT was created directly from Sveasoft’s software decision to start charging for their firmware, closing the door to opensource.
DD-WRT comes in various flavors, including reduced-feature builds for newer routers with only 2 MB of flash and versions that add VoIP and VPN features to an impressively large base feature set. It supports routers based on Broadcom or Atheros processors.
Tomato is a relative newcomer, positioning itself as "a small, lean and simple replacement firmware". It supports only Broadcom-based routers, in particular, Linksys’ WRT54G/GL/GS and Buffalo WHR-G54S/WHR-HP-G54.
The Router
I didn’t want to spend time finding about all the things that can go wrong with uploading alternative firmware. So I chose the Linksys WRT54GL (v1.1) as my test platform. It has 16 MB of RAM and 4 MB of flash. It was created by Linksys in response to the outcry from the alternative firmware crowd when Linksys cut RAM and flash in half on the WRT54G starting with V5.0.
Figure 1: Linksys WRT54GL
I looked at both the WRT54G V5.0 and WRT54GL about two years ago and found the WRT54G better than the WRT54GL in terms of throughput consistency, but the GL better for simultaneous session handling.
The Test – Simultaneous Up and Downlink
So I set out to see if either DD-WRT or Tomato would smooth out the GL’s high throughput variation and raise the maximum number of simultaneous sessions that it would support. My first step was to load up the current firmware release (4.30.12) and rerun my standard set of router tests.
Figure 1 shows that the high throughput variation that I saw with Linksys version 4.30.2 firmware is still there. (You can see the orignal IxChariot plot here.)
Figure 2: Linksys WRT54GL simultaneous up / down throughput with Linksys 4.30.12 firmware
The results with DD-WRT v23sp2 loaded are shown in Figure 3. It looks like DD-WRT allows uplink and downlink throughput to not battle each other as it does in the factory firmware. But when both up and downlink traffic are simultaneously high, uplink gets most of the bandwidth.
Note that I checked that the DD-WRT QoS features were disabled and also tried the SPI firewall disable. Neither changed the imbalance between up and downlink throughput.
Figure 3: Linksys WRT54GL simultaneous up / down throughput with DD-WRT v23sp2
Figure 4 shows the same simultaneous up and downlink test run with Tomato 1.19 loaded. While the behavior is different than that of the factory firmware, I don’t think that you can say that it’s better.
Figure 4: Linksys WRT54GL simultaneous up / down throughput with Tomato 1.19
The Test – Separate Up and Downlink
I next ran each firmware in downlink and uplink only. Figure 5 shows a download throughput comparison of the three firmwares and Figure 6 the upload.
Figure 5: Linksys WRT54GL download throughput comparison
The good news is that Tomato appears to improve up and downlink throughput by around 14%. But remember that Tomato didn’t fix the battle between up and downlink when both are running.
On the other hand, while DD-WRT makes simultaneous up and downlink play nice, it appears to do it at the expense of throughput, with a reduction of about 34% in both directions.
Figure 6: Linksys WRT54GL upload throughput comparison
The Test – Maximum Simultaneous Connections
The last test I ran was the maximum simultaneous connection check. Unfortunately, I have little good news to report. Figure 7 shows the test run with factory firmware. I was able to get only 48 maximum connections to run, and that took the maximum of three tries that the test allows.
Note the high variation in throughput in Figure 7, which, along with the high variance in completed timing records (file transfers), indicates unequal bandwidth sharing among the sessions.
Figure 7: Maximum simultaneous connection test – Linksys 4.30.12 firmware
Figure 8 shows the DD-WRT results. You see much lower variation among the sessions, which indicates good bandwidth allocation. But this better behavior didn’t significantly boost the maximum number of sessions, which increased to only 60 maximum simultaneous connections.
Figure 8: Maximum simultaneous connection test – DD-WRT v23sp2 firmware
Finally, Figure 9 shows the Tomato results, which have a high variation that doesn’t bode well for a high maximum connection number. In fact, I was able to get only 16 maximum connections to run, and that, only after multiple unsuccessful tries.
Figure 9: Maximum simultaneous connection test – Tomato 1.19 firmware
Conclusion
In case you haven’t been keeping score, I’ve pulled the results into Table 1.
| Firmware | Simultaneous Up/Dn Throughput (Mbps) |
Dn Throughput (Mbps) |
Up Throughput (Mbps) |
Max. Simultaneous Connections |
|---|---|---|---|---|
| Linksys 4.30.12 | 28.2 | 50.2 | 47.2 | 48 |
| DD-WRT v23sp2 | 31.0 | 31.6 | 30.4 | 60 |
| Tomato 1.19 | 19.1 | 57.4 | 53.7 | 16 |
| Winner | DD-WRT | Tomato | Tomato | DD-WRT |
Table 1: Test result summary
The bottom line is that while both DD-WRT and Tomato improve the WRT54GL’s routing performance, neither one improves performance in all four of the tests performed. But if I had to choose, I would load up DD-WRT if I were having routing problems.
While it only provides a modest improvement in maximum simultaneous connections, the fact that it seems to keep sessions from battling one another should make for more stable routing performance.