Per packet overhead on VDSL2 – part 2

1 Reply

A few days ago I wrote about some interesting latency results I observed on my home Internet connection with small packets. This post adds a bit more data.

In this experiment I disabled all upstream traffic shaping and then used iperf to blast UDP packets of various sizes to a destination host I control. The transmitted rate was 10Mbps and the upstream link rate is ~6.5Mbps. On the destination I captured the packets with tcpdump and generated the charts below with Wireshark.

The charts show ten sub-experiments – 10 seconds of traffic for each data size (iperf -l): 25, 50, 75, 100, 200, 300, 400, 500, 1000, 1400 bytes. T0 is when the first packet is received.

Packets per second observed at the destination

The first chart shows the packets per second received at the destination. Not surprisingly, the packet rate is much higher with small packets.

Bitrate observed at the destination

The second chart shows the bitrate observed at the destination. Notice that for small packets the effective bitrate is much lower. This seems to support the theory that this link has a lot of per-packet overhead.

Per packet overhead on VDSL2

7 Replies

My home router (Linux box) is configured to shape upstream traffic to just below the link rate to avoid Bufferbloat – this greatly improves interactive performance under load. Recently I’ve experimented with various packet sizes. The charts below show the effect of small packets.

Effect of per-packet overhead on VDSL2?

  1. Between 0-6 seconds the link is idle.
  2. From 6-14 seconds the upstream link is flooded with 1,400 byte packets (10Mb/sec of traffic trying to get through a 6.2Mb link)
  3. At 17 seconds the upstream link is flooded with 64 byte packets (10Mb/sec as well)

Notice how much higher the latency and jitter are with small packets.

Confusingly, these results were gathered with the bandwidth shaper configured for 53 bytes of overhead [1] which is my current understanding of the per-packet overhead on VDSL2 [53 is a coincidence with the ATM cell size].

Per-packet overhead for VDSL2 (without ATM) and PPPoE:

  • 5 Bytes for PTM
  • 40 bytes for 802.3
  • 8 bytes for PPPoE

Either the above overhead numbers or wrong or there is something else going on.

[1] – overhead argument to tc.

 

Measuring one way packet loss

1 Reply

I’ve recently observed packet loss on my home Internet connection and suspected that this was occurring on the downstream path. Unfortunately, ping, the tool normally used to measure packet loss doesn’t tell you anything about where the packet was lost – just that either the request to the destination or the reply from the destination was lost.

After a quick bit of Googling I couldn’t find a tool which measured one way packet loss so I fell back to combining a couple of tools. Hopefully the technique outlined here will be useful to others.

Note that the steps below require that you have control of both of the hosts you wish to measure packet loss between. It’s very easy and cheap to get a virtual machine on Amazon EC2 or other services so this shouldn’t be a huge barrier.

To measure packet loss on the path from host A to host B run the following commands.

Host B

tcpdump -nni ppp0 udp port 8787 > tmp.file

Host A

hping3 -i u100000 --destport 8787 -2 HOSTB

Let some time elapse and then press CTRL-C to stop hping3.

Host B

Press CTRL-C to stop tcpdump

cat tmp.file | egrep 8787 | wc -l

Now compare the number of UDP packets transmitted by hping3 to the number of packets captured by tcpdump (output of wc -l). If there is any difference there is packet loss from A to B.