Re: [Openvpn-users] Windows client, mysterious routes and MTU issues

["John A. Sullivan III" <jsullivan@xxxxxxxxxxxxxxxxxxx>]

Please pardon the top post but I'll use the previous body as additional
information to this post.  We now know what the problem is but it's
uglier than we thought.  We are getting severe fragmentation on our
OpenVPN network.  Large, do-no-fragment pings work perfectly fine until
we launch a TCP session.  Then the tunnel MTU plummets to 576.  The
problem is PMTU discovery.  Either we have something very badly
misconfigured (configuration files are below) or we have hit a bug (far
less likely!).

The bottom line is that, on a tunnel with a perfectly fine MTU as we can
see from the 1472 byte pings, OpenVPN is returning a fragmentation
needed ICMP packet for any sized TCP packet and, worse yet, it says that
the acceptable MTU is zero bytes! (Unless I am misreading my ethereal
traces).

Here's what I saw and what I tried:

We established that we could ping the full 1500 byte MTU with:
ping -f -t -l 1472 10.7.254.1 - all replies successful until we launched
an ssh session with the same host.  The TCP sequence began as follows:

No.Time  Packet Size Source   Destination   Protocol Info
1 0.000000  62     172.20.98.2  10.7.254.1  TCP      1349 > 22 [SYN]
Seq=0 Ack=0 Win=16384 Len=0 MSS=1460

2 0.049058  90     172.20.98.1  172.20.98.2 ICMP     Destination
unreachable (Fragmentation needed)

Notice the very small packet size. This was only a 62 byte packet.  MTU
is 1500.  Worse yet, here is the expansion of the ICMP packet returned
by the OpenVPN server:

Internet Control Message Protocol
    Type: 3 (Destination unreachable)
    Code: 4 (Fragmentation needed)
    Checksum: 0x133e [correct]
    MTU of next hop: 0
    Internet Protocol, Src: 172.20.98.2 (172.20.98.2), Dst: 10.7.254.1
(10.7.254.1)
    Transmission Control Protocol, Src Port: 1349 (1349), Dst Port: 22
(22), Seq: 2813985907, Ack: 0

Notice the MTU next hop! Is that normal? I would assume the Windows
default behavior when receiving such a request is to go to the old
default MTU of 576.  That's what we are seeing.

I wondered if the bogus MTU might be coming from some other source as
this is a fairly complex setup.  The path is from the tun device to the
local eth device to the eth device on the OpenVPN gateway to the tun
device of the gateway to the ipsec0 device on the gateway to the eth on
the gateway to the eth on the remote network gateway to ipsec0 to eth,
etc.  The problem could be anywhere so we traced at multiple points.

The do not fragment appears to be coming from the OpenVPN server.  We
see the ICMP frag-needed packets on the gateway tun device.  We do not
see them on ipsec0 (we even check lo just in case), so they do not
appear to be reflected back from a downstream problem (I'm not even sure
if these messages are reflected backwards).

Thus, it seems that the OpenVPN gateway advertising an MTU of 0 is the
problem.  If we disable PMTU discovery on the Windows client, all works
well and there is no fragmentation.

The configuration files are included below from the original post.  Have
we misconfigured our systems or stumbled across a bug? We are using
OpenVPN 2.1 beta 7 on the client and beta 8 on the server.  The server
side is running in a Xen domU (virtual machine) running Fedora Core 3.
Thanks, all - John




On Sat, 2006-02-04 at 18:02 -0500, John A. Sullivan III wrote:
> Hello, all.  We are seeing some bizarre behavior with our Windows
> OpenVPN clients.  We are not seeing the same behavior in the Linux
> clients.  Our setup is an OpenVPN gateway running openswan connecting to
> other offices via openswan.  I am just now beginning to peruse the more
> obscure configuration settings and do some extensive testing but, if
> anyone knows the answer off the top of their head, it will save me many
> hours.
> 
> Whenever the windows client accesses a station in one of the offices,
> i.e., across the openswan VPN, an entry for that station is added to the
> Windows routing table.  Although that is surprising, it would not be too
> bad except that the MTU on the route is set to 576! The result is
> massive packet fragmentation.
> 
> This happens without or without using the fragment directive.  The
> routing entries are not made when using a Linux client.  So, for
> example, when the windows client first connects, they can do:
> 
> ping -f -l 1472 x.x.x.x
> 
> After a moment, the route is added and the same command fails until the
> length is reduced to 548 or lower.
> 
> We are using 2.1 beta8.  The client config is:
> 
> client
> dev tun
> proto udp
> remote x.x.x.x 1194
> resolv-retry infinite
> nobind
> ca /etc/openvpn/certs/NiagaraCA.pem
> pkcs12 /etc/openvpn/certs/jsullivan.atlas.p12
> ns-cert-type server
> tls-auth tlsauthniag 1
> comp-lzo
> verb 3
> #fragment 1435
> #mssfix
> passtos
> 
> The server config is:
> 
> port 1194
> proto udp
> dev tun0
> key /etc/ipsec.d/private/niagararasgwk.pem
> cert /etc/ipsec.d/certs/niagararasgwc.pem
> ca /etc/ipsec.d/cacerts/NiagaraCA.pem
> crl-verify /etc/ipsec.d/crls/NiagaraCRL.pem
> dh /etc/openvpn/certs/dh5.pem
> server 172.20.100.0 255.255.255.128
> topology subnet
> ifconfig-pool-persist ipp.txt
> client-connect /etc/openvpn/clientconn.script
> client-disconnect /etc/openvpn/clientdisconn.script
> push "redirect-gateway def1"
> push "dhcp-option DNS 10.4.1.77"
> push "dhcp-option DNS 10.1.1.22"
> push "dhcp-option DNS x.x.x.x"
> push "dhcp-option WINS 10.1.1.10"
> push "dhcp-option WINS 10.4.1.35"
> keepalive 10 120
> tls-auth /etc/openvpn/tlsauthniag 0 # This file is secret
> comp-lzo
> persist-key
> persist-tun
> status /var/log/openvpn/openvpn-status.log
> log-append  /var/log/openvpn/openvpn.log
> verb 3
> passtos
> management 127.0.0.1 1194 /etc/openvpn/mgmtfile
> 
> The tricky thing with these kinds of problems is that everything
> actually works and no one would know the worse if we hadn't seen the
> fragmentation messages.  Now that we know, we know we can make a
> dramatic improvement in performance if we can fix this.  Thanks - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@xxxxxxxxxxxxxxxxxxx

If you would like to participate in the development of an open source
enterprise class network security management system, please visit
http://iscs.sourceforge.net


____________________________________________
Openvpn-users mailing list
Openvpn-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/openvpn-users