TCP Offloads: more harm than good

+Dave Taht has been saying for a while that TCP offloads do more harm than good, especially when mixed with fq_codel, and the ingress rate limiter that the sqm_scripts package uses to replace the large inbound buffers in the modem and CMTS with a much smaller buffer (but nearly as fast bandwidth), under the control of the router.

I finally put some numbers on that tonight.

The first dataset (green plots) are without gro, tso, and gso.  The second plots are with those offloads all re-enabled.  So enabling offloads:

1) slows it down
2) increases latency

??

Yeah, I'm keeping all the offloads turned off (and adjusted my router startup scripts to keep them off each time the simple.qos script runs).