iperf3 and microbursts, part II

Previously, I talked about iPerf3's microbursts, and how changing the frequency of the timer could smoothen things out.

One of the first feedback items that I got, and lined up with some musings of my own, was if it might be better to calculate the timer based on the smoothest possible packet rate:

PPS = Rate / Packet Size

This is, btw, what iPerf3 does automatically on Linux, when the fq socket scheduler is available.  So this is really just seeing if we can fake it from user-land on systems that don't have it (like my OSX system)

Luckily, adding this to iPerf's code is pretty easy.

iperf3 and microbursts

This is in the "Always know how your tools work" category.

Background

We were doing some end-to-end network testing at work using iperf3.  While TCP would happily fill the link, UDP was giving us just miserable results.  It wouldn't achieve more than 8 mbps of 'goodput' on a 50mbps link (which we'd just verified using TCP).  Extra packets over 8mbps were getting dropped.

The setup at the traffic generation end was:

PC -> switch1 -> switch2 -> 50mbps link to ISP

• PC to switch1 was 1Gbps
• switch1 to switch2 was 100Mbps

If we connected the PC running iperf3 directly to switch2, we'd get much better throughput (or, rather, much less packet loss for the same traffic load).  But then the PC was transmitting into a 100Mbps port, not a 1Gbps port...

I thought that this sounded like perhaps packet bursts were exceeding some buffers, and got curious as to how iperf3 actually generates traffic and performs pacing with the -b option.

Packet Bursts

I personally didn't have access to the test setup, so I did some quick code inspection.  What I found was that it turns transmission on using a 100ms timer, turning transmission back off when it's achieved the right long-term average transmit rate (specified with -b ).

What this ends up looking like for a 100Mbps rate transmitting into a much higher rate interface, using 1KB packets, is below.  This is a graph of MB send per 1ms period, scaled into Mbps.

Here's the first second of that, using the same Y scale (1Gbps):

So yeah, bursty.  Not exactly "micro" bursts, either.  More like millibursts.