[bittorrent] using bittorrent for cluster file distribution
Arnaud.Legout at sophia.inria.fr
Fri May 27 04:12:53 EDT 2005
I just have some comments inline.
First, when talking about network throughput, I will use the notation
kb/s meaning kbit per seconds, and I will understand
KB/s as kilobytes per seconds.
Ryan Stenson wrote:
> I’m a consultant at the Cornell Theory Center (www.tc.cornell.edu
> <http://www.tc.cornell.edu/>). We’re considering using BitTorrent to
> move large files from a file server/job submit node to the compute
> nodes (1:M). Using multicast, we get about 30-40 Mb/s, however our
> network (GigE) should be capable of 800+. Does anyone have similar
> experience with this sort of application?
There is no reason that would limit a MC transfert to 40Mb/s in an ideal
case (no bottleneck, high throughput, low loss rate).
The performance depends on the MC protocol you use. MC file transfert is
a complex problem
that depends on your scenario. There is no MC protocol that fit all the
cases. In your case, I assume that you have
not bottleneck bandwitdth and a uniform set a machines. So source based
or receiver based MC protocol should fit your needs.
Some possible reasons of performance problems:
-you use a source based (TCP friendly) MC protocol, and one machine or
the path to one machine has poor performance. In this
case that will impact all your session.
-you use a receiver based layered MC protocol
*your aggregate layers throughput does not exceed 40Mbit/s.
*your layers are well chosen, but the heuristic to choose which layer to
subscribe to does not work accurately in your environment.
Note that I am simplifying a lot the problem.
> I've setup Bittorrent on a few servers in an attempt to speed up our
> current multicast file distribution system (we do a lot of copying
> from a few fileservers to dozens of clients).
Do you mean that you setup one torrent per server, all clients
subscribing to all the torrents?
> I'm only seeing 20 KB/s transfer speeds over Gigabit ethernet (these
> machines are dual xeon's, 4gb ram, etc.). We see 30-40 Mb/s over
> multicast, which is
on a pentium M laptop 1.4Ghz, with the same python client as you, with
the same startup mode (btdownloadheadless) I reach around 4000kb/s
(500KB/s) with a 50% CPU load.
You can try to increase the min_upload_rate to a high value, e.g. 1000,
by default it is 20 kBytes/s.
> extremely slow compared to our theoretical throughput of 800+ Mb/s.
> Are there any default settings that are limiting my throughput? Any
> other suggestions? I realize that using the start bittorrent code,
> uncompiled, isn’t the optimal solution, but this is just the first step.
I do not believe that your problem comes from the optimization of the
code (at least, as long as you do not reach several Mb/s)
More information about the BitTorrent