|
|
|
Hi
!
I've written a
software that transfers the nightly backups to another location using
NetBIOS (Windows File- and printer sharing) through a Open VPN-connection.
We're talking massive amounts of data each night, which introduces a problem:
Transfer speed.
At first I had a
single thread that transfered all the files, but then I only got about 12% in
average of the max speed, which translates to 1.2 MBit/s. The
transferspeed-curve was also quite jagged.
So I rewrote some of
the software, to make it multithreaded. After adding a second thread, I got
it up to around 25%, but after that, adding more threads really doesn't make
much of a difference - it only makes the transferspeed-curve a bit smoother
for each thread added.
And here's the
really weird part: I get around 25% per server. So if I transfer two files
from one server, and two files from another server, it hits around 50%. 6
files from three servers makes it go to around 75%. This is true regardless
of wether I use my own software, or if I just use the file explorer to transfer
the files.
So it seems that the
NetBIOS-protocol limits the amount of connections to one, and
the added amount of transfer-threads maximizes the performance of that
single connection. So my four threads have to share that single
NetBIOS-connection, resulting in a stunted performance. At least that's my
enterpertation of the problem. I know this might not be all that
OpenVPN-related, but any hints on how to boost NetBIOS-performance by beeing
able to make more than one connection (or some other optimization), would
be greatly appreciated. I know for a fact that IIS is able to make one
NetBIOS-connection / file that it monitors (which wreaked havoc with our
fileserver when setting up our new Web Farm, causing about 500 simultaneous
NetBIOS-connections (and Windows had a default limit at around 150 or somehting
like that)), so it should be possible to do this programatically. I just dont
know how.
Right now, the only
solution I can think of, would be to create 3 virtual machines, all sharing the
backup-drives, which would make me able to transfer the data using 4
connections, resulting in a maximized transferspeed. But that seems like a
really bad solution (plus the fact that it would be difficult to make the
software handle it in a good way, since it would see it as four separate
backup-sets, and download each file four times, resulting in the same
performance, or worse, than I have today)
Regards
Henric
Rosvall
|