[bittorrent] 'Rarest first' question.

Bruno Hertz brrhtz at yahoo.de
Mon Jul 3 14:09:30 EDT 2006

Arnaud Legout <Arnaud.Legout at sophia.inria.fr> writes:

>> Please find the words "must" anywhere in the mentions of rarest first.
>> Random order is *any* order, at which point any selection is valid by
>> the protocol. Notice too, that portion is untestable. You can look at the
>> over the wire and .torrent files and deterministically state that
>> something is valid or not. You cannot test for a random selection order
>> and so any is valid.
> I am not sure I am interpreting correctly what you are saying. If you 
> intended to point out
> that as you cannot test from a client the piece selection strategy of 
> other clients, you cannot
> specify with a MUST (note that I am not shouting, I am just using the 
> notations used in RFCs)
> the piece selection strategy, you are wrong. Specifying a protocol does 
> not mean that you can enforce the
> specification. Take for instance TCP, you can enforce the packet header 
> format, you cannot enforce the
> congestion control algorithm.
> (The reality a bit more complex, but the details are not relevant for 
> this discussion)
> One criteria to deem if a protocol is good is when a selfish user must 
> follow the protocol specification in order to improve
> its the satisfaction.
> This is clearly the case of rarest first when your satisfaction comes 
> from the speed of download.
> My understanding is that this notion of random pieces comes from the 
> first specification of the protocol by Bram Cohen.
> Then he figured out that random piece selection leads to the last pieces 
> problem.

I think the question about the original motivation(s) for introducing
rarest first is answered by Bram's posting here:

> Thus he implemented rarest first, but he never updated
> his specification (which is a documentation rather than a specification).

It really has to be emphasized how unfortunate it is we haven't even
seen an attempt on an RFC during the five years BT is now

Also, by closing that discussion group Bram apparently cut himself off
the community, which imho wasn't a great move either. Especially with
regard to implementors who are seeking to comply but might have

>> That may be the effect, but I highly doubt it was the original goal. As
>> already pointed out, beyond a certain level of entropy, additional
>> entropy isn't useful. Once every piece is helf by at least six of your
>> neighbors, performance of those neighbors is more crucial than entropy.
> and what happens if you have a flash crowd after the start of the 
> torrent. Finding a relevant threshold
> is hard and depends on the context (torrent size, mean number of pieces, 
> etc.)
> There are two important properties of BT:
> -efficient content replication
> -scalability/resilience to flash crowd
> For both properties rarest first is required.

One might add to your reasoning that entropy resp. shareability
apparently was a focus from the beginning, cf. the link above:

"Rarest first downloads pieces starting at the least common. These are the
pieces which both are most likely to be unavailable later and are most
likely to be useful for uploading to peers."

especially the phrase "to be useful for uploading to peers".

> I agree that there are tons of scenarios for which rarest first may not 
> be appropriate, but in this
> case does it still makes sense to consider BT. We can surely reuse part 
> of it, but if you don't use rarest first
> or the choke algorithm, then it is not BT anymore.

Regards, Bruno.


Gesendet von Yahoo! Mail - Jetzt mit 1GB Speicher kostenlos - Hier anmelden: http://mail.yahoo.de

More information about the BitTorrent mailing list