Cake: the latest in sqm (QoS) schedulers

Today I finally had the opportunity to try out Cake, the new replacement for the combination of HTB+fq_codel that the bufferbloat project developed as part of CeroWRT's sqm-scripts package.

Background

The bufferbloat project is tackling overbloated systems in two ways:

1. Removing the bloat everywhere that we can
2. Moving bottlenecks to places where we can control the queues, and keep them from getting bloated

sqm-scripts, and now cake, are part of the latter.  They work by restricting the bandwidth that flows through an interface (ingress, egress, or both), and then carefully managing the queue so that it doesn't add any (or much) latency.

More details on how cake works can be read HERE.

The WNDR3800

Cake was meant to perform well on lower-end CPUs like those in home routers.  So the test results that follow are all on a Netgear WNDR3800.  This was a fairly high-end router, 5 years ago when it was new.  Now, it's dual 802.11n radios are falling behind the times, and it's 680MHz MIPS CPU is distinctly slow compared to the >1GHz multi-core ARM CPUs that are currently in many home routers.

All the tests that follow were taken using the same piece of hardware.

Final Results

I'm starting with the final results, and then we'll compare the various revisions of settings and software that led to this.

Comcast Service Speeds:

180Mbps download

12Mbps upload

100s of ms of latency

Cake's shaping limits (before the CPU is maxed out):

~135 Mbps download speed

12Mbps upload

no additional latency vs idle conditions

What's really impressive is how smooth the incoming streams are.  They really are doing well.  Upstream is also pretty good (although not great, this is the edge of what the CPU can manage).  But what's simply amazing is the latency graph.  It doesn't change between an idle or fully-in-use connection.

And the CDF plot really shows that.  There's no step between the idle and loaded operation, just a near-vertical line around the link latency (which is almost entirely between the modem and the head-end).

Gear Review: Brevite Camera Backpack

As I mentioned in my previous post, while the Crumpler bag is great for travel, it's not so great for day-to-day use, and in particular isn't great on a hike.  But then, neither are most hiking back-packs, being designed to be out of the way when you're hiking.

So when I saw the the Brevite kickstarter, I decided to take a chance and back it, and see if it would work out better for me.

Unfortunately, it's not well suited to me and my cameras.  But I think it would be fine for many other people.  An important clue in all of this was in the kickstarter campaign, and had I been paying closer attention, would have realized the issue I ran into in advance.

But first let's talk about the bag, and what it does well.

Gear Review: Crumpler Proper Roady Photo Full Backpack

3 years ago, when living in Paris, I needed something for carrying camera gear on planes that was better than my 10yo daypack.

So I ordered this, having seen the Amazon Basics knock-off version of it:

Proper Roady Photo Full Backpack, by Crumpler

It's been utterly fantastic for carting camera gear and a latop on planes (or any other time I need to travel with the "full kit".  It holds a ton of camera gear and a 15" laptop, fits under-seat if it needs to, and is fairly comfortable once all the straps are adjusted right.

Here it is:

iperf3 and microbursts, part II

Previously, I talked about iPerf3's microbursts, and how changing the frequency of the timer could smoothen things out.

One of the first feedback items that I got, and lined up with some musings of my own, was if it might be better to calculate the timer based on the smoothest possible packet rate:

PPS = Rate / Packet Size

This is, btw, what iPerf3 does automatically on Linux, when the fq socket scheduler is available.  So this is really just seeing if we can fake it from user-land on systems that don't have it (like my OSX system)

Luckily, adding this to iPerf's code is pretty easy.

iperf3 and microbursts

This is in the "Always know how your tools work" category.

Background

We were doing some end-to-end network testing at work using iperf3.  While TCP would happily fill the link, UDP was giving us just miserable results.  It wouldn't achieve more than 8 mbps of 'goodput' on a 50mbps link (which we'd just verified using TCP).  Extra packets over 8mbps were getting dropped.

The setup at the traffic generation end was:

PC -> switch1 -> switch2 -> 50mbps link to ISP

• PC to switch1 was 1Gbps
• switch1 to switch2 was 100Mbps

If we connected the PC running iperf3 directly to switch2, we'd get much better throughput (or, rather, much less packet loss for the same traffic load).  But then the PC was transmitting into a 100Mbps port, not a 1Gbps port...

I thought that this sounded like perhaps packet bursts were exceeding some buffers, and got curious as to how iperf3 actually generates traffic and performs pacing with the -b option.

Packet Bursts

I personally didn't have access to the test setup, so I did some quick code inspection.  What I found was that it turns transmission on using a 100ms timer, turning transmission back off when it's achieved the right long-term average transmit rate (specified with -b ).

What this ends up looking like for a 100Mbps rate transmitting into a much higher rate interface, using 1KB packets, is below.  This is a graph of MB send per 1ms period, scaled into Mbps.

Here's the first second of that, using the same Y scale (1Gbps):

So yeah, bursty.  Not exactly "micro" bursts, either.  More like millibursts.