[BitTorrent] Re: bt2 protocol features

Elliott Mitchell ehem at m5p.com
Thu Jul 15 19:38:28 EDT 2004


>From: Olaf van der Spek <OvdSpek at LIACS.NL>
> Elliott Mitchell wrote:
> >>From: John Prevost <j.prevost at gmail.com>
> > Another group. Any structural changes to one effects the other. If hash
> > mode was used, ordering by the arithmetic order of the hashes makes more
> > sense, otherwise no real comment. I like grabing a byte in the BITFIELD
> > message to designate the hash-type in use, as opting for SHA256 is a
> 
> Shouldn't that be put in the .torrent instead of in the BITFIELD msg?

For the torrent yes, but that isn't the only place where that information
needs to be. With a multi-file peer protocol you might be handling
torrents that use different hashes, so the protocol packets would need to
identify which was in use (yeah I glossed over this).


>From: John Prevost <j.prevost at gmail.com>
> Your point about TCP overhead is well taken--and I'm really not sure
> what can be done about that.  The problem is that "non-contributing"
> peers are still important--for one thing, a peer which isn't your best
> peer at time t1 might end up being a best peer (and sharing a lot of
> data with you) at a later time t2.  For another, I can certainly
> imagine that a peer with a very low bandwidth connection might end up
> *only* transferring data to and from other peers in the optimistic
> unchoke slot--it would never transfer quickly enough to become one of
> the favored unchokes of any peer, but it would still be contributing
> and still be receiving data.

I think this suggests that clients should be fairly aggressive about
dropping unused peers, if things change they can reconnect later. You do
want to keep some peers around just in case your active ones disconnect,
but you don't want that number to be large. Looking at the statistics, it
looks like the mainline look for 35 peers is pretty reasonable, while the
general number cited for bandwidth aproximations, 100, is pretty
ridiculous. I think clients should take steps to strongly discourage
users from changing it as it won't help them.

> Perhaps one option to reduce overhead would be to think about how the
> protocol would have to change to make HAVE messages bufferable.  For
> example, if the protocol is still workable without knowing which peers
> are interested in you, it might be possible to send one big HAVE when
> unchoking a peer.  Chances are good, except at the very start and very
> end of a file, that every peer is interested in every other peer.  If
> you optimistically assume that, I don't think it does anything bad. 
> It would just mean that you occasionally unchoke someone who won't
> actually request any data from you.  (And the WANT variety of messages
> makes this a non-issue for seeds who are never interested.  They would
> UNWANT any file that was completed, or never WANT it in the first
> place if connecting as a peer.)

I doubt this approach will be useful. Your IP stack should deal with
buffering fairly well. The problem is the raw number of HAVE messages,
not their size.

> Oh, and a couple of specific comments:
> 
> > Another group. Any structural changes to one effects the other. If hash
> > mode was used, ordering by the arithmetic order of the hashes makes more
> > sense, otherwise no real comment. I like grabing a byte in the BITFIELD
> > message to designate the hash-type in use, as opting for SHA256 is a
> > foreseeable future choice. Given the size of the HAVE message, you can
> > infer the size of the hash, and differing hashes aren't likely to
> > collide.
> 
> I think ordering by arithmetic order in the hashes doesn't make
> sense--this is a Merkle hash tree, remember?  That means that when we
> start out, we don't *know* the individual piece hashes.  And it's most
> likely that hash requests will be somewhat smaller than "give me every
> single piece hash".  For example, if I'm requesting the first piece, I
> only need the hashes for the right subtrees all the way down the tree.

Gets back to why I was originally proposing by hash mode.  :-)  Mainly to
be able to remove reassembly information from helpers. Arithmetic order
is useful here because it is unrelated to the order of the pieces in the
reassembled file. Interesting question of how these work together.

I actually suspect the Merkle trees are mostly useful at the lowest
level. Mainly always distribute all the Merkle hashes of the pieces, and
only conditionally distribute the block hashes.

One option might be to distribute an index into the bitfield with the
hashes. There are weaknesses here, but this is at least possible.

> My example of GETHASH was really a pretty lame example, just to show
> one really naive (and broken, honestly) way to do it.

For distributing piece hashes I think that is a pretty accurate
description.  :-)  For retrieving block hashes after a piece has been
found to be defective it is reasonable. With the piece hashes the problem
is you're going to need too many of them for it to be worthwhile to
retrieve them 8 at a time. Figure you'll likely want to retrieve a
minimum of 1024 at a time.

> This is the thing we really need Bram's input on--exactly how the
> Merkle tree and protocol are meant to interact.

Yup. Ideally a preliminary protocol proposal so we can see if we can
suggest refinements.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM at gremlin.m5p.com PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/




------------------------ Yahoo! Groups Sponsor --------------------~--> 
Make a clean sweep of pop-up ads. Yahoo! Companion Toolbar.
Now with Pop-Up Blocker. Get it for free!
http://us.click.yahoo.com/L5YrjA/eSIIAA/yQLSAA/dkFolB/TM
--------------------------------------------------------------------~-> 

 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 



More information about the BitTorrent mailing list