[bittorrent] IO bound

Carsten Otto c-otto at gmx.de
Fri Apr 13 11:50:47 EDT 2007


On Fri, Apr 13, 2007 at 12:46:40PM +0200, Adrian Ulrich wrote:
> Hello Otto,

Carsten is my first name.

> > Long story short, delivering torrents from the disk is no good solution
> > for me.
> 
> So what storage system are you using? Raid5 ? Raid0 ?

I personally use a software RAID 5 over four disks. I have acess to many
other systems, most of them with some sort of RAID 1 or no RAID at all,
a few with RAID 5. In some cases I could reconfigure the storage. So the
actual disk layout should be of no concern, no matter what it is too
slow for 1 or 10 GBit/sec of (random) access.

> IMO having 2 Raid0 systems would be good enough for BT as BT handles
> redundancy itself: If one raid0 blows up, you can symple re-sync it
> >from other existing peers (aka your 2nd raid0 bt-system)

During the first tests with the 12 disk system I noticed that the
controller (3Ware 7xxx) slows down RAID X when RAID Y is heavily
utilized. Apart from that I did not see a difference in speed between
RAID 0 and RAID 1 (with 3 disks). Storing the data in a safe place is no
problem - I could redownload them via bittorrent in the worst case.

> Bittorrent will always do many small and random read()/write()'s and
> behaves almost like an smtp/imap server. This kind of stuff will always
> be IO-Bound and the only real solution is to buy more RAM / Better
> Disk-System (Buy a DMX-3 and you won't have any more problems ;-) )

No, I believe in the force of algorithms!! :)

> So if you are serving too much data (with not-so-much RAM/Disk-Cache)
> you are doomed (= IO/Bound)

That is the current situation.

> However you can try to reduce 'randomness', serving only data from RAM.
> I wouldn't mess with Suggest/FastPeers .. i'd just send the clients
> a fake bitfield:

I know that concept (I suggested it in my first mail). The discussion in
this ML is quite interesting, I am following.

> This should work without disconnecting existing clients: Just send
> HAVE messages to them.

They will continue to download the chunks that were in the old cache.

> (Well: 'old' clients may still ask for a currently-not-shared chunk
>  because BitTorrent does not implement a DONT_HAVE_ANYMORE message.

Correct.

>  But this shouldn't be a big problem because this clients had a
>  chance to download the piece a long time ago and also it should
>  be well seeded now. But in the worst case you could also simply
>  disconnect existing clients before switching to a new
>  fake bitfield)

Every chunk is well seeded. I am talking about high throughput torrents,
not some initial seed. Disconnecting (a lot of) clients is a realistic
and worst case.

> Using bitflu (http://bitflu.workaround.ch) you'd just need to change ~ line 1770 to send
> this fake bitfields (and implement sending out HAVE messages for existing clients if you are switching)
> 
> 1770: _xwA($sock, Bitflu::Protocol::CreateBitfield($TORRENT_UNIVERSE->{$tref->{INFO_HASH}}->{bitfield}));

Good to know. My current opinion though is to give virtually all clients
a correct implementation of the already mentioned "suggest" thing - if I
have the time, which I doubt.

Bye,
-- 
Carsten Otto
c-otto at gmx.de
www.c-otto.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://lists.ibiblio.org/pipermail/bittorrent/attachments/20070413/a6a77c3d/attachment.bin 


More information about the BitTorrent mailing list