[bittorrent] using bittorrent for cluster file distribution

Ryan Stenson rms48 at cornell.edu
Thu May 26 16:06:52 EDT 2005


Hi all,
 
I'm a consultant at the Cornell Theory Center (www.tc.cornell.edu
<http://www.tc.cornell.edu/> ).  We're considering using BitTorrent to move
large files from a file server/job submit node to the compute nodes (1:M).
Using multicast, we get about 30-40 Mb/s, however our network (GigE) should
be capable of 800+.  Does anyone have similar experience with this sort of
application?
 
I've setup Bittorrent on a few servers in an attempt to speed up our current
multicast file distribution system (we do a lot of copying from a few
fileservers to dozens of clients).  
 
I start the tracker with:              bttrack.py --port 6969 --dfile dstate
I then create the torrent with:     btmaketorrent.py http://IP:6969/announce
c:\temp\filename
and seed the torrent with:          btdownloadheadless.py
\\server\filename.torrent --save_as c:\temp\filename
and download it to the clients with the same command.
 
I'm only seeing 20 KB/s transfer speeds over Gigabit ethernet (these
machines are dual xeon's, 4gb ram, etc.).  We see 30-40 Mb/s over multicast,
which is extremely slow compared to our theoretical throughput of 800+ Mb/s.
Are there any default settings that are limiting my throughput?  Any other
suggestions?  I realize that using the start bittorrent code, uncompiled,
isn't the optimal solution, but this is just the first step. 
 
Thanks in advance!
 
Ryan Stenson
Cornell Theory Center
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ibiblio.org/pipermail/bittorrent/attachments/20050526/88885b7e/attachment.html 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3024 bytes
Desc: not available
Url : http://lists.ibiblio.org/pipermail/bittorrent/attachments/20050526/88885b7e/attachment.bin 


More information about the BitTorrent mailing list