[bittorrent] 'Rarest first' question.

Arnaud Legout Arnaud.Legout at sophia.inria.fr
Mon Jul 3 05:26:57 EDT 2006


Elliott Mitchell wrote:
> Please find the words "must" anywhere in the mentions of rarest first.
> Random order is *any* order, at which point any selection is valid by
> the protocol. Notice too, that portion is untestable. You can look at the
> over the wire and .torrent files and deterministically state that
> something is valid or not. You cannot test for a random selection order
> and so any is valid.
I am not sure I am interpreting correctly what you are saying. If you 
intended to point out
that as you cannot test from a client the piece selection strategy of 
other clients, you cannot
specify with a MUST (note that I am not shouting, I am just using the 
notations used in RFCs)
the piece selection strategy, you are wrong. Specifying a protocol does 
not mean that you can enforce the
specification. Take for instance TCP, you can enforce the packet header 
format, you cannot enforce the
congestion control algorithm.
(The reality a bit more complex, but the details are not relevant for 
this discussion)
One criteria to deem if a protocol is good is when a selfish user must 
follow the protocol specification in order to improve
its the satisfaction.
This is clearly the case of rarest first when your satisfaction comes 
from the speed of download.

My understanding is that this notion of random pieces comes from the 
first specification of the protocol by Bram Cohen.
Then he figured out that random piece selection leads to the last pieces 
problem. Thus he implemented rarest first, but he never updated
his specification (which is a documentation rather than a specification).

> That may be the effect, but I highly doubt it was the original goal. As
> already pointed out, beyond a certain level of entropy, additional
> entropy isn't useful. Once every piece is helf by at least six of your
> neighbors, performance of those neighbors is more crucial than entropy.
and what happens if you have a flash crowd after the start of the 
torrent. Finding a relevant threshold
is hard and depends on the context (torrent size, mean number of pieces, 
There are two important properties of BT:
-efficient content replication
-scalability/resilience to flash crowd

For both properties rarest first is required.

I agree that there are tons of scenarios for which rarest first may not 
be appropriate, but in this
case does it still makes sense to consider BT. We can surely reuse part 
of it, but if you don't use rarest first
or the choke algorithm, then it is not BT anymore.


More information about the BitTorrent mailing list