Version 4 (modified by, 5 years ago) (diff)


Why do I see poor bandwidth on my stitched links with iperf?

Answer (provided by Nick Bastin):

The window size is way too small if you want single-flow throughput to be high, particularly given the latency in WAN connections (the resultant bandwidth delay product will be much too high for iperf's default window sizes).

You simply need to set the window size larger on both the server and client and your performance should be better for a single flow, regardless of raw-pc or xen VM. (The default raw PC window size is likely a lot larger, hiding this problem, but if you want deterministic performance from iperf you always need to set the window sizes you're going to use, otherwise it makes a guess at a default that is going to be highly intolerant of any latency in your connection).

Barring any other shaping going on, the throughput you're getting with a 23.5K window means you have a connection with roughly 12ms of delay. To get 100 Mbits (or close to it) on this connection, you'd need a window size of 150K or larger (and make sure your buffer sizes on both end can support that large of a window).

Brecht Vermeulen adds: typically also handy in debugging link performance is to throw in an UDP iperf to see what is possible on the links and when packet loss begins to start: server side: iperf -s -u client side: iperf -c xxx -b 50M

so you can vary bandwidth and packet size and see what happens

For the whole thread that lead to this discussion see:

Should I use a raw PC for my experiment?

Many aggregates in GENI have only a couple of raw PCs and thus are a scarce resource, since when a user reserves one, no one else can use them. Moreover, some AMs have policies in place that do not allow a user to renew slivers with raw PCs for a long time.

When you are designing your experiment only include a raw PC if you absolutely need to. In most cases you can do everything you want with a Virtual Machine (VM). This way you will leave the raw PCs to experimenters that really need them. If you are not sure if a VM is appropriate for your experiment don't hesitate to send an email to

Why do I only see traffic on one interface on my multi-homed system?

When does this apply? You have a multi-homed system (i.e. it has multiple data plane interfaces) which appears to only be able to receive traffic from a given traffic source on one interface.

What's happening? Reverse path filtering may be enabled. We most often see this issue on Ubuntu systems because reverse path filtering is enabled by default.

How to fix it On Ubuntu 12.04 run the following command:

sudo sysctl -w net.ipv4.conf.all.rp_filter=0