[bittorrent] 'Rarest first' question.

Arnaud Legout Arnaud.Legout at sophia.inria.fr
Wed Jun 28 16:46:26 EDT 2006


Bruno Hertz wrote:
> I browsed through your paper, thanks very much for the link. Two
> questions though:
> (i)  since you measured torrents in the wild, how could you actually be
>      sure the other clients employed rarest first? Not only did the
>      original BT spec recommend random piece selection, but it may also
>      be that developers twist the algorithms to serve other goals.
no, the BT spec urges to use the rarest first piece selection. There is 
a random piece selection for the first pieces only.
I give some details on the piece selection in the paper.
>      E.g. the two first projects I contacted these recent days looking
>      for a BT library I could use are considering implementing resp.
>      have already implemented hybrid algorithms, with partial rarest
>      first and partial sequential piece selection. One project to
>      allow to watch movies while downloading, the other to provide for
>      better disk caching.
I am not sure what you mean by hybrid. It is known that random piece 
selection performs poorly
in most cases and do not outperform rarest first. You can have a random 
choice when you have a set
of pieces that are rarest. Also, if you have pieces ordering constraints 
(live stream, content preview)
you may implement something else than rarest first. But this is at the 
cost of a decrease in system capacity/efficiency.
>      From this I got the impression clients might do what they want
>      anyway, so did you actually limit your measurements to peers you
>      know for sure use rarest first?
I have the client ID for all the peers in the torrents I monitored. To 
the best of my knowledge everybody implement rarest
first. There are some specificities from protocols to protocols, but the 
main piece selection strategy (i.e., the piece selection
strategy used for most of the download) is always rarest first.
We have checked the code of several clients in several versions (in 
particular Azureus and mainline).
You are right that clients can implement something else. But, there is 
no known piece selection strategies that performs better
than rarest first (except if you have coding techniques, which is not 
the case for my experiments). So there is no incentive
to use something else than rarest first.
> (ii) the graphic plots in the pdf are pretty hard to read. You haven't
>      some larger copies lying around on some web server, haven't you? I'd
>      really be interested in seeing some more detail.

I can send you the original eps version of any plots if you give me the 
fig number in the paper.
I which details are you interested in.
> Finally, while browsing through this list's archives, I saw you
> mentioning you might make your instrumented client publicly
> available. There isn't any news in that regard, is there?
I have not yet put it in the public domain. However, if you have a 
specific experiment in mind, we can discuss it.


More information about the BitTorrent mailing list