[bittorrent] using bittorrent for cluster file distribution

Justin Cormack justin at street-vision.com
Wed Jun 1 07:14:10 EDT 2005


On 27 May 2005, at 02:17, larytet.8708132 at bloglines.com wrote:

> "I didnt do any real performance tests, although on the same host  
> with a large
> file, moving it to the same hard drive I got around 20
> MB/s, so one should
> be able to do much better between machines."
>
> do you have the source code
> opened ? i have tough problems with thread schedulers in the OSs. i  
> found
> out that its close to impossible to "sleep" for less than 10ms, for  
> example.
> it would be very interesting at least for me to discuss different  
> implementations
> of rate limiters.
>
> in my tests i reached 1500Kbyte/s (see for example http:// 
> larytet.sourceforge.net/images/tests/7clients_1300K.PNG)
> when running 7 clients on the same single CPU system. my current  
> implementation
> of the data trasnfer layer divides all available bandwidth by slots  
> of about
> 50Kbytes/s (or less). it means that to reach 15M between two peers  
> running
> on regular Windows host (all threads in Rodi client run in low  
> priority, besides
> socket Rx) i need ~30 up/down slots. In the terms of the Data  
> Transfer layer
> it means 30 simultaneous streams, which means 30 sessions. Session  
> in Rodi
> is a transaction of one block. Default block size is 4M. lifetime  
> of the session
> < 2min. Naturally Rodi can not max up 15M upstream  if file size is  
> less than
> 30 blocks or 120M.
> the good news are that Rodi is UDP based and multicast
> support is possible. Still there is no doubt that ways exist to  
> improve performance
> in the client
>
> greatly appreciating any implementation details/ideas, arkady
>

I havent looked at rate limiting yet, but it would be a good thing to  
discuss. I was hoping that with tcp it might be relatively straight  
forward as you can watch the tcp buffers emptying, but I havent  
really tried yet. Maybe I will do some test code.

Doing scheduling in userspace doesnt sound good.

Anyone else have any thoughts?

j




More information about the BitTorrent mailing list