Re: [Openvpn-users] Windows client, mysterious routes and MTU issues

["John A. Sullivan III" <jsullivan@xxxxxxxxxxxxxxxxxxx>]

On Mon, 2006-02-06 at 17:05 +0100, BlaaT 0001 wrote:
> I did some reading and did find the reason for your Next Hop MTU = 0 question.
> 
> source: http://www.networksorcery.com/enp/protocol/icmp/msg3.htm
> ________________________________________________________
> Next-Hop MTU. 16 bits.
> >From RFC 1191 for use when the code is set to 4: when a router is
> unable to forward a datagram because it exceeds the MTU of the
> next-hop network and its Don't Fragment bit is set, the router is
> required to return an ICMP Destination Unreachable message to the
> source of the datagram, with the Code indicating "fragmentation needed
> and DF set". To support the Path MTU Discovery technique specified in
> this memo, the router MUST include the MTU of that next-hop network in
> the low-order 16 bits of the ICMP header field that is labelled
> "unused" in the ICMP specification. The high-order 16 bits remain
> unused, and MUST be set to zero....The size in bytes of the largest
> datagram that could be forwarded, along the path of the original
> datagram, without being fragmented at this router. The size includes
> the IP header and IP data, and does not include any lower-level
> headers. This field will never contain a value less than 68 since
> every router must be able to forward a 68 byte datagram without
> fragmentation.
> ______________________________________________________
> 
> Correct me if I'm wrong, but what I understand "The high-order 16 bits
> remain unused, and MUST be set to zero" means a Next-Hop-MTU of 0
> follows. So after a "ICMP Destination Unreachable message" a Next Hop
> MTU = 0 follows.

Thanks very much.  I think your understanding is just slightly off.  The
high order bits set to 0 just mean that portion of the field is still
unused.  The next hop MTU is in the low order bits.  In fact, in a test
system I now have working, I see OpenVPN returning proper values in the
MTU field (in this case 1400) and not 0.
> 
> Source: http://www.tcpipguide.com/free/t_IPDatagramSizetheMaximumTransmissionUnitMTUandFrag-4.htm
> 
> IP Datagram Size, the Maximum Transmission Unit (MTU), and
> Fragmentation Overview
> (Page 4 of 4)
> 
> ____________________________________________________
> Internet Minimum MTU: 576 Bytes
> 
> Each router must be able to fragment as needed to handle IP datagrams
> up to the size of the largest MTU used by networks to which they
> attach. Routers are also required, as a minimum, to handle an MTU of
> at least 576 bytes. This value is specified in RFC 791, and was chosen
> to allow a "reasonable sized" data block of at least 512 bytes, plus
> room for the standard IP header and options. Since it is the minimum
> size specified in the IP standard, 576 bytes has become a common
> default MTU value used for IP datagrams. Even if a host is connected
> over a local network with an MTU larger than 576, it may choose to use
> an MTU value of 576 anyway, to ensure that no further fragmentation
> will be required by intermediate routers.
> 
> Note that while intermediate routers may further fragment an
> already-fragmented IP message, intermediate devices do not reassemble
> fragments. Reassembly is done only by the recipient device. This has
> some advantages and some disadvantages, as we will see when we examine
> the reassembly process.
> ________________________________________________________________
> 
> This explains the MTU of 576 value on your Windows clients.

Yes, if Windows cannot do PMTU discovery or gets a very small MTU back,
it will default to this lowest common denominator.

> 
> 
> What still questions me is why the Windows clients are creating routes
> for traffic with a destination behind the IPSEC tunnel. Is this
> because of an ICMP Redirect Gateway packet?
> 
> This is what I understand from your description of your lay-out.
> 
> OpenVPN client <-> OpenVPN server <-> IPSEC endpoint left <-> IPSEC
> endpoint right <-> Remote LAN PC
> 
> How does the OpenVPN server route the traffic to the IPSEC tunnel?
> Through the TUN/TAP device? If the OpenVPN client and the IPSEC
> endpoint left reside on the same segment your router might decide to
> send an ICMP Redirect Gateway to the OpenVPN client telling him how to
> reach the Remote LAN. Because of the ICMP Destination Unreachable
> message a Next Hop MTU = 0 is translated to a minimal MTU of 576 and
> added to the routing table of your Windows clients.

That's an interesting conjecture.  Thank you.  I am assuming that
OpenVPN is dropping of the packets from the tun device, they pass
through the filter table and to the nat table where KLIPS then reroutes
them to ipsec and the ipsec interfaces.

The openswan end point and the client do not reside on the same subnet.

I am finding out much more than I ever wanted to know ;) The route entry
happens whenever the OpenVPN server sends a fragmentation needed packet
to the client but only when the fragmentation point is the ipsec tunnel.
For example, if I set the physical interface MTU to 1400 but leave the
ipsecmtu at its default, the OpenVPN client receives a proper PMTU
discovery from the remote side and adjusts is MSS without the routing
entry.  I am assuming that the ipsec interface receives the packet
without complaining to OpenVPN since it's MTU is fine.  When it cannot
send over the physical interface, it fragments it's packet since it's
packet was not sent to do not fragment.  The other side reassembles the
do not fragment packet.  The destination responds with the required MSS.
This packet is stuffed into ESP, fragmented back to the OpenSWAN ipsec
interface, reassembled and sent back unfragmented to OpenVPN.  At least,
I think that's what must be happening.

If I set the physical MTU to 1500 but reduce the ipsecmtu to 1400, the
OpenVPN server rather than the destination point returns a fragmentation
needed and a route is added in the client for the destination address.
In this case, the ipsec interface cannot "fake" the do-not-fragment.
Again, I am assuming that's the case.

I do not know why OpenVPN causes the route to appear.  This happens in
both beta7 and beta8.  However, in my test lab, the MTU is properly
returned as 1400.  In production, the MTU is returned as 0.  I thought
this might be a beta8 problem as the lab was on beta7 but an upgrade in
the lab produces the same results - MTU=1400.

So, I'm stumped on two points:

1) Why is the route added?
2) Why does my production system bungle the MTU message?

It's been almost two days of head banging so if anyone has any ideas,
I'd greatly appreciate them.  I'll post the resolution when we finally
find it so we can spare someone else this grief :)  Thanks, all - John

> 
> I might be entirely wrong or I might have misunderstood you. But right
> now it's 17:00h and time to go home so I really don't have the time to
> read through all your posts on this matter.
> 
> Good luck anyhow and I hope this helps you.
> 
> Cheers,
> 
> Jasper
> 
> 
> 
> On 2/6/06, Erich Titl <erich.titl@xxxxxxxx> wrote:
> > John A. Sullivan III wrote:
> > ..
> > >
> > > <snip>
> > > The Windows client requests an MSS of 1460:
> > > Transmission Control Protocol, Src Port: 1349 (1349), Dst Port: 22 (22),
> > > Seq: 0, Ack: 0, Len: 0
> > >     Source port: 1349 (1349)
> > >     Destination port: 22 (22)
> > >     Sequence number: 0    (relative sequence number)
> > >     Header length: 28 bytes
> > >     Flags: 0x0002 (SYN)
> > >     Window size: 16384
> > >     Checksum: 0x7b77 [correct]
> > >     Options: (8 bytes)
> > >         Maximum segment size: 1460 bytes
> > >         NOP
> > >         NOP
> > >         SACK permitted
> >
> > OK, this is the syn packet
> >
> > >
> > > Then the OpenVPN server replies saying the MTU is 0:
> > >     Type: 3 (Destination unreachable)
> > >     Code: 4 (Fragmentation needed)
> > >     Checksum: 0x133e [correct]
> > >     MTU of next hop: 0
> >
> > This is a bit premature, can you correlate this packet with the previous
> > TCP request? What are the source and destination addresses for this packet.
> >
> > >
> > > Then destination replies that it wants an MSS of 1368:
> > > Transmission Control Protocol, Src Port: 22 (22), Dst Port: 1349 (1349),
> > > Seq: 0, Ack: 1, Len: 0
> > >     Source port: 22 (22)
> > >     Destination port: 1349 (1349)
> > >     Sequence number: 0    (relative sequence number)
> > >     Acknowledgement number: 1    (relative ack number)
> > >     Header length: 28 bytes
> > >     Flags: 0x0012 (SYN, ACK)
> > >     Window size: 32440
> > >     Checksum: 0x364d [correct]
> > >     Options: (8 bytes)
> > >         Maximum segment size: 1368 bytes
> > >         NOP
> > >         NOP
> > >         SACK permitted
> >
> > And this baffles me, as it appears to be the correct SynAck Packet for
> > the TCP request. So my main question is, why do you get a SynAck for a
> > connection which was rejected beforehand with an ICMP type 3
> > (destination unreachable) message? For my understanding there is
> > something amiss here.
> >
> > >
> > > Why am I getting an MTU request of 0 from the OpenVPN server? 1500 byte
> > > pings pass fine.  I would expect that PMTU-D should work:
> >
> > Mhhh.... your ICMP messages may not have the DF bit set....
> >
> > cheers
> >
> > Erich
> >
> >
> >
> > -------------------------------------------------------
> > This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
> > for problems?  Stop!  Download the new AJAX search engine that makes
> > searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
> > _______________________________________________
> > Openvpn-users mailing list
> > Openvpn-users@xxxxxxxxxxxxxxxxxxxxx
> > https://lists.sourceforge.net/lists/listinfo/openvpn-users
> >
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@xxxxxxxxxxxxxxxxxxx

Financially sustainable open source development
http://www.opensourcedevel.com


____________________________________________
Openvpn-users mailing list
Openvpn-users@xxxxxxxxxxxxxxxxxxxxx
https://lists.sourceforge.net/lists/listinfo/openvpn-users