[bittorrent] Introductory/endgame algorithms

Elliott Mitchell ehem at m5p.com
Wed Sep 28 19:22:16 EDT 2005


>From: Jari Sundell <sundell.software at gmail.com>
> On 9/27/05, Elliott Mitchell <ehem at m5p.com> wrote:
> > You get major perform damage if you keep 100 peers alive. The number of
> > HAVE messages is directly related to the number of peers. At 30 peers the
> > HAVE messages account for 50% of the BitTorrent protocol overhead, or 1%
> > of the payload size. At 100 peers, HAVE messages are accounting for 75%
> > of the overhead, 3% of the size of the payload.
> >
> > Though 3% isn't a huge percentage, considering the size that payloads
> > run, 3% is likely to be several megabytes. Do you see a reason that
> > justifies an additional 2% overhead?
> 
> Some of us have bandwidth enough to actually need more than 100 peers,
> having written clients that regularly upload to hundreds of peers at speeds
> up to 50MB/s. I never manage to saturate my bandwidth, and if the torrent
> has enough peers I try to upload to as many as possible to maximize my
> download speed.

Never saturate your bandwidth? Which direction? Sounds like you've
roughly got a decent strategy, upload to many peers (rather than the 4-6
that the mainline defaults to) so that many peers favor you and you get
enough bandwidth in aggregate. I must question whether the number of
peers is truely a limitation here. Unless you've preloaded everything in
memory, very few disks will support 50MB/s random I/O. I *will* conceed
that you're into the region where more peers may be needed.

> Bandwidth is cheap, time isn't. A little overhead from HAVE messages and
> keepalives doesn't hurt, especially when using epoll.

True. My point though is the circumstances under which more peers will in
fact provide additional performance are quite rare. Gigabit speed
Internet connections are rare, and even when present they tend to be
shared and you're unlikely to be able to monopolize 50% of it.

> I'm limiting the number of peers that can download a single piece to 5 in
> the endgame mode. With a 2 minute timeout this has shown itself to work well
> in all cases I've encountered. Also the piece with fewest concurrent
> downloads is delegated next. Peers with a transfer speed less than 4kb/s
> only queue a single piece, while faster peers use a queue size of half what
> they do in normal downloads.

Uh, you need to clear this paragraph up. You're saying that you'll only
request blocks from a particular piece from a grouping of 5 peers?

The second though is a bad idea. If the queue depth on the remote end is
ideal (zero or near zero), then you've just cut your download rate by
50%.


>From: Jari Sundell <sundell.software at gmail.com>
> On 9/28/05, Olaf van der Spek <olafvdspek at gmail.com> wrote:
> > If there's enough RAM, can't you simply prefetch all files completely into
> > RAM?
> 
> I haven't considered doing this explicitly, I use mmap'ed files directly and
> the kernel keeps the pages in memory if there's room. If the user wants to
> preload the files, he may use dd or similar to dump them into /dev/null.

Bad idea to do it that way. I doubt most OSes do read-ahead on mmap()ed
files, and for what BitTorrent is doing readahead is quite important.
Your best bet is use pread()/readv() on whole pieces when the first block
is requested, and ensure your buffer is page-aligned
(sysconf(_SC_PAGESIZE)).

There are two key points here. First, when the first block of a piece is
requested, very probably subsequent blocks will be read (I'd even advise
adding a seek penalty each time a new piece is requested to prevent
deliberate attacks). By reading the whole piece you'll avoid a second
seek returning to fetch the rest of the piece, a crucial performance
factor. Second, by doing I/O to page-aligned boundaries the OS is free to
do copies by merely memory-maping the files in and doing zero-copy.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         EHeM at gremlin.m5p.com PGP 8881EF59         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/





More information about the BitTorrent mailing list