[bittorrent] 'Rarest first' question.
Arnaud.Legout at sophia.inria.fr
Mon Jul 3 05:26:57 EDT 2006
Elliott Mitchell wrote:
> Please find the words "must" anywhere in the mentions of rarest first.
> Random order is *any* order, at which point any selection is valid by
> the protocol. Notice too, that portion is untestable. You can look at the
> over the wire and .torrent files and deterministically state that
> something is valid or not. You cannot test for a random selection order
> and so any is valid.
I am not sure I am interpreting correctly what you are saying. If you
intended to point out
that as you cannot test from a client the piece selection strategy of
other clients, you cannot
specify with a MUST (note that I am not shouting, I am just using the
notations used in RFCs)
the piece selection strategy, you are wrong. Specifying a protocol does
not mean that you can enforce the
specification. Take for instance TCP, you can enforce the packet header
format, you cannot enforce the
congestion control algorithm.
(The reality a bit more complex, but the details are not relevant for
One criteria to deem if a protocol is good is when a selfish user must
follow the protocol specification in order to improve
its the satisfaction.
This is clearly the case of rarest first when your satisfaction comes
from the speed of download.
My understanding is that this notion of random pieces comes from the
first specification of the protocol by Bram Cohen.
Then he figured out that random piece selection leads to the last pieces
problem. Thus he implemented rarest first, but he never updated
his specification (which is a documentation rather than a specification).
> That may be the effect, but I highly doubt it was the original goal. As
> already pointed out, beyond a certain level of entropy, additional
> entropy isn't useful. Once every piece is helf by at least six of your
> neighbors, performance of those neighbors is more crucial than entropy.
and what happens if you have a flash crowd after the start of the
torrent. Finding a relevant threshold
is hard and depends on the context (torrent size, mean number of pieces,
There are two important properties of BT:
-efficient content replication
-scalability/resilience to flash crowd
For both properties rarest first is required.
I agree that there are tons of scenarios for which rarest first may not
be appropriate, but in this
case does it still makes sense to consider BT. We can surely reuse part
of it, but if you don't use rarest first
or the choke algorithm, then it is not BT anymore.
More information about the BitTorrent