[OpenVPN home] [Date Prev] [Date Index] [Date Next]
[OpenVPN mailing lists] [Thread Prev] [Thread Index] [Thread Next]
Web openvpn.net

Re: [Openvpn-users] Multicast fragmenting not handled by OpenVPN ?

  • Subject: Re: [Openvpn-users] Multicast fragmenting not handled by OpenVPN ?
  • From: "James Yonan" <jim@xxxxxxxxx>
  • Date: Sun, 1 Jun 2003 05:01:52 -0000

James MacLean <macleajb@xxxxxxxxxxx> said:

> On Sat, 31 May 2003, James Yonan wrote:
> > James MacLean <macleajb@xxxxxxxxxxx> said:
> > > Hi Folks,
> > > Wanted to report that openvpn- does properly get the multicast 
> > > packets through using the suggestion from below. It's the only tunnel that 
> > > does it.
> > Great, I'm glad it's working for you.
> Well... :) It was/is working... At work, between 2 machines both with 
> linux 2.4.20 type kernels and using the openvpn tun interfaces for mrouted 
> to route over (no mrouted own tunnel involved).
> But from work to home, which is from a 2.2.x kernel to a 2.4.20 ish 
> kernel, it does not appear to be doing its magic. Maybe I should not have 
> enabled pthread in openvpn...

I doubt that pthread is an issue here.  It doesn't really do anything unless
you are negotiating the tunnel with TLS, though it wouldn't hurt to rule it
out by building without pthread.

> Also, from home to work it does mrouted 
> tunneling over the openvpn tunnel. So it is a little different, but all I 
> see coming through tun0 and dropping onto eth1 is a pile of DF packets, 
> and all not near the 1470 that get created at the other end. So, something 
> is a miss along the way.

I've never used mrouted, though I understand that it uses IP over IP tunnels,
and so is likely using IP fragmentation as well.  If you look at the source
snippet below, you will see that if the linux kernel encounters a multicast
packet larger than the MTU that the destination can support, it drops it into
a black hole.  The IP over IP code appears to take the path MTU of the
destination referenced by the top-level encapsulation and subtract the IP over
IP header size to derive the MTU of the inner encapsulation.  I would guess
that someone trying to send a packet to the endpoint of an IP over IP tunnel
will cause the kernel to see this lesser MTU and fragment according to the
fragmentability of the particular IP protocol that's being tunneled.  In the
cast of multicast, it could end up routing a large packet to /dev/null, since
multicast doesn't fragment.

OpenVPN's internal fragmentation, on the other hand, blindly fragments every
packet larger than the max-size specified in the --mtu-dynamic option.

> > The typical usage of --mtu-dynamic would be a scenario where you want to
> > bridge an ethernet segment using a TAP virtual device, but where IP
> > fragmentation is broken on the tunnel path.  Because ethernet uses a fixed MTU
> > of 1500, when you encapsulate a 1500 byte ethernet frame into an encrypted UDP
> > tunnel packet, the added overhead of encryption and encapsulation will push
> > the resulting packet size to >1500 which will therefore require fragmentation.
> >  If IP fragmentation is working between the hosts, the following would be
> > sufficient:
> > 
> > openvpn --dev tap --up ./mktap --remote [other-openvpn-host] --secret key
> > --tun-mtu 1500 --tun-mtu-extra 64 --comp-lzo
> > 
> > where mktap is a script:
> > 
> > # local = local IP endpoint for tunnel
> > ifconfig $1 $local netmask mtu $2
> > 
> > Then you would need to bridge $1 (which will be something like tap0) with your
> > ethernet LAN, using a tool such as linux's brctl.
> And this certainly doesn't work. Not one packet makes it off eth1 after
> coming through the tunnel that was originally large (ie 1470 bytes). But
> again, this is along the lines of what I am commenting on right from the
> start. The packets all hit the first end of the tunnel broken up with the
> DF set (and in this non-working case I'm gonna be bold a blame it on the
> kernel as it is mrouted's tunneling that is sending it's payload over the 
> tunnel) wosh through the tunnel out the other end and onto eth1, DF still 
> set and the packets crippled. Bandwith suffering with a max around 270Kbs. 
> It's so bad the streaming app (mp4live) doesn't perform.

Can you configure mrouted to use openvpn tunnels directly, rather than routing
an IP over IP tunnel through the openvpn tunnel?

I suspect (as I believe you do) that the IP over IP tunnel might be killing
the large multicast packets before they ever reach the local endpoint of the
openvpn tunnel.

> > On the other hand, if IP fragmentation is broken between the hosts, adding
> > "--mtu-dynamic 1450 1450" will cause OpenVPN to internally fragment packets so
> > that the tunnel UDP datagrams never exceed 1450 bytes (not including the IP
> > header).  This fixes the problem because UDP datagrams <=1450 will usually not
> > trigger IP fragmentation. 
> This one works at work. Doesn't work to home, but I am strongly starting 
> to believe its mrouted's tunneling, which is using the kernel that is 
> causing my grief...
> > > I did not get any response to my last E-mail with regards to what OpenVPN 
> > > and/or the kernel is supose to be doing with the normal settings when a 
> > > large packet has to be squeezed through the tunnel. If it should get 
> > > fragmented, it does not appear to be setting up the packets as fragmented 
> > > packets. If it is supose to break them up to get through the tunnel, that 
> > > also does not appear to be happening, as whatever is sliced up in the 
> > > tunnel ends up on the outgoing ethernet also sliced up.
> > When you say "normal settings" I assume you mean without --mtu-dynamic. 
> Yes.
> > Now in a multicast setting, Path MTU discovery isn't going to work because
> > multicast is by nature a one-way communication flow.  As you can see in the
> > linux kernel snippet below, linux will refuse to send an ICMP message in
> > response to a multicast packet.  However, if you are tunneling a multicast
> > stream through an OpenVPN tunnel, that stream will be tunnelled as a series of
> > UDP packets which can then benefit from Path MTU discovery and IP
> > fragmentation as it is implemented for the UDP layer.
> Sure, the UDP packets of the tunnel can negotiate MTU... But still the 
> underlying MultiCast is not being helped out.

How large are the multicast packets?  Do you have any control over their size?

> > > I am hoping someone can explain this as it appears to be common to OpenVPN
> > > in it's normal mode, GRE tunnels, and mrouted's tunnel. Also, the overall
> > > effect here is to slow traffic leaving the outgoing ethernet down to
> > > around 270Kbs, as if it is causing the kernel to get stalled.
> > Bottom line is that most tunnels will break if you try to bridge ethernet
> > segments over a tunnel where Path MTU Discovery and IP Fragmentation are not
> > working correctly.  tcpdump is a good tool to ascertain whether this is
the case.
> Then as a bottom line :), if a packet arrives at a tunnel and that packet
> is bigger than the tunnel can deal with, then the kernel can not properly
> break it up into fragments and send them through the tunnel. Openvpn can 
> :).
> If it is just a packet traveling between normal interfaces (eth0, etc...)
> it will be properly fragmented and passed through by the kernel.
> So bottom line is if you have any applications that send packets through
> the tunnel where the packets are bigger than what the tunnel can handle as
> a single packet, they will be somehow crippled and sent through the tunnel
> with the DF set and cause the tunnel to max sending this large payload at
> about 270 Kbs. Atleast for MultiCasts packets.
> Can't say I would expect Linux to behave this way.

Well here's the linux source (2.4.19).  You not only cannot fragment
multicasts, but you cannot report your inability to do so via the ICMP mechanism.

if (skb->len+encap > rt->u.dst.pmtu && (ntohs(iph->frag_off) & IP_DF)) {
	/* Do not fragment multicasts. Alas, IPv4 does not
	   allow to send ICMP, so that packets will disappear
	   to blackhole.

> > > Happy camper now to see 1 tunnel solution that will work with large 
> > > packets effectively ;).
> > Excellent!
> Except now I have to find a way to remove any kernel tunneling apps that 
> might be in use :(.
> -- 
> James B. MacLean        macleajb@xxxxxxxxxxx
> Department of Education 
> Nova Scotia, Canada
> B3M 4B2


Openvpn-users mailing list