[bittorrent] IO bound

Adrian Ulrich torrent at blinkenlights.ch
Fri Apr 13 06:46:40 EDT 2007


Hello Otto,

> Long story short, delivering torrents from the disk is no good solution
> for me.

So what storage system are you using? Raid5 ? Raid0 ?

IMO having 2 Raid0 systems would be good enough for BT as BT handles
redundancy itself: If one raid0 blows up, you can symple re-sync it
from other existing peers (aka your 2nd raid0 bt-system)

> Is there a client or at least some well-thought algorithm that solves
> this problem?

Bittorrent will always do many small and random read()/write()'s and
behaves almost like an smtp/imap server. This kind of stuff will always
be IO-Bound and the only real solution is to buy more RAM / Better
Disk-System (Buy a DMX-3 and you won't have any more problems ;-) )

So if you are serving too much data (with not-so-much RAM/Disk-Cache)
you are doomed (= IO/Bound)

However you can try to reduce 'randomness', serving only data from RAM.
I wouldn't mess with Suggest/FastPeers .. i'd just send the clients
a fake bitfield:

Let's say you've got 1gb RAM and you are going to seed a 5gb torrent.

Normally your client would send such a bitfield:

[XXXXXXXXXXXXXXXXXXXXXX]  (X = Have piece | . = Missing)
   ^--- total content = 5gb

..but because you do not have enough ram for 5gb you'll send
a fake bitfield:

[XXXX..................]
  ^--- total content =~ 1.7gb

So you'll only get requests for stuff you can keep in your cache.

After some time (= pieces are well seeded.. degraded upload.. etc)
you can send a different fake bitfield:

[.X..XXX................]
  ^-- total content =~ 1.7gb

This should work without disconnecting existing clients: Just send
HAVE messages to them.

(Well: 'old' clients may still ask for a currently-not-shared chunk
 because BitTorrent does not implement a DONT_HAVE_ANYMORE message.
 But this shouldn't be a big problem because this clients had a
 chance to download the piece a long time ago and also it should
 be well seeded now. But in the worst case you could also simply
 disconnect existing clients before switching to a new
 fake bitfield)


Using bitflu (http://bitflu.workaround.ch) you'd just need to change ~ line 1770 to send
this fake bitfields (and implement sending out HAVE messages for existing clients if you are switching)

1770: _xwA($sock, Bitflu::Protocol::CreateBitfield($TORRENT_UNIVERSE->{$tref->{INFO_HASH}}->{bitfield}));


Regards,
 Adrian


-- 
 RFC 1925:
   (11) Every old idea will be proposed again with a different name and
        a different presentation, regardless of whether it works.




More information about the BitTorrent mailing list