[Openvpn-users] NFS over OpenVPN over PPPoE: MTU and performance problems

[Moritz Bunkus <m.bunkus@xxxxxxxxxxxxxxxxx>]

Hey,

I've got a customer whose three locations we have connected with
OpenVPN. Now they're migrating their servers to Linux, and we're
stumbling into problems with NFS. Let's just consider two locations, A
and B, which are connected to the internet with DSL (PPPoE with an
interface MTU of 1492). On top of that we use OpenVPN. There's a Linux
file server in each location.

However, as soon as we mount a NFS share from server A on server B and
try to copy any file that's bigger than lets say 1 KB the NFS share
stalls, and the only thing I can do is to "umount -f" it.

My first try was to lower the wsize/rsize, and that does indeed
work. However, they can only be set to multiples of 1024 (or the kernel
rounds down automatically), so 1024 is the only value that didn't cause
the connection to stall. The problem is that this severely impacts
performance. After Googling a bit I've also tried using NFS-over-TCP
which worked tons better, but it did require a recompilation of the
kernel (which is why I've only done that on the test servers, not on the
production servers). Here are some measurements of copying one large
file from one server to the other (run multiple times, of course, each
time with umount/mount inbetween):

Method | wsize/rsize | Duration (seconds)
-------+-------------+-------------------
UDP    | 1024/1024   | 122
UDP    | 1450/1450   | 122
TCP    | standard    | 42

So I thought "well let's give OpenVPN's fragmentation a try" because I'd
prefer not to use NFS-over-TCP because it has undergone far less testing
than NFS-over-UDP on Linux and because of that kernel recompilation
issue (which would also create problems with vendor support I
guess). But that didn't work quite like I thought it would.

At first I only tried using "fragment 1450" and "mssfix" on both
ends. Here's the config file (static keys for simplicity, again from my
test systems):

------------------
dev tun
remote 172.16.50.3
ifconfig 10.0.0.1 10.0.0.2
route 192.168.51.0 255.255.255.0
secret r.key
port 1194
keepalive 5 15
persist-tun
persist-key
verb 3

#tun-mtu 1480
fragment 1450
mssfix
------------------

But NFS-over-UDP still stalls with standard parameters (meaning without
setting wsize/rsize to 1024). NFS-over-TCP is still working.

Next I tried adding "tun-mtu 1400" and changing "fragment" to 1300, just
to be on the safe side. Same effect. Each time I've made the change on
both ends of the connections and restarted OpenVPN.

So I'm guessing that I'm interpreting the effect of those options
wrong. Or maybe I'm interpreting them right, but there's something else
going on here...

Any hints? Back in another thread about NFS-over-OpenVPN someone said
he'd have no problems with that setup by adjusting the application
packet size (rsize/wsize for NFS), but as I'm seeing such a performance
loss I don't want to go that road.

Thanks for any tips.

Regards,
Moritz Bunkus

-- 
LINET Services GbR

Gotenweg 15                      Tel.: 0531-280 191 71
38106 Braunschweig               Fax.: 0531-280 191 72

http://www.linet-services.de
mailto:info@xxxxxxxxxxxxxxxxx

Attachment: pgpcpMseYY5CL.pgp
Description: PGP signature