[bittorrent] Friend-machines-assisted approach to boosting one's BT download

Elliott Mitchell ehem at m5p.com
Mon Jun 12 00:06:45 EDT 2006


>From: Harold Feit <dwknight at depthstrike.com>
> Elliott Mitchell wrote:
> >> From: Harold Feit <dwknight at depthstrike.com>
> >> Elliott Mitchell wrote:
> >>>> From: Harold Feit <dwknight at depthstrike.com>
> >>>> Elliott Mitchell wrote:
> > 
> > Yes, how does this damage the peer or swarm?
> > 
> > The non-cluster peer can still download those pieces from another
> > non-cluster peer. Unless a single meta-peer is at least 40 times a swarm
> > it joins, I don't see it causing any sort of problems. Crucially there is
> > little point in employing a meta-peer larger than around half the size of
> > a swarm because at that point, the meta-peer's bandwidth will likely
> > outperform the swarm's, at which point additional members won't increase
> > download speed.

> With a meta-peer that has not downloaded the piece, that interaction is
> forced rather than voluntary, causing a negative skew in piece distribution.

How?

> > At least with my thoughts, the individual slaves are still act pretty
> > similarly to other peers. They would still retrieve what appear to be
> > rarer pieces, since other local peers are likely to want to download
> > those, in exchange for being able to download other pieces they still
> > desire.
> > 
> > As a /group/ the meta-peer still downloads one copy, and has to upload
> > roughly one copy. Yes, that copy is broken into distinct sets scattered
> > across the swarm, but this is nearly an ideal situation for seeding. Any
> > pieces not held by a local member of the cluster will be obtained via
> > other peers. The swarm still holds at least two distributed copies, one
> > on the meta-peer and one across all the peers the meta-peer downloaded it
> > from.
> > 
> > Where is the damage?

> When the meta-peer leaves, they take the ENTIRE meta-peer's COMBINED
> upload rate AND distribution away from the swarm.
> In smaller swarms, this can be fatal to the piece distribution for the
> given torrent.

Yes, what about people on OC12s leaving swarms though? These two are
pretty comparable, a node with large amounts of bandwidth leaving a swarm
will be noticable, but not fatal.

Small torrents are always in jeopardy. Do you tell someone on an OC12 to
never leave a swarm?

> >>> Again, slaves and the master conform to the BitTorrent protocol. If they
> >>> can harm the swam by conforming to the protocol, then the protocol needs
> >>> to be fixed. I don't see how they would harm the swarm though.
> >> If they are selectively not downloading pieces that others in their
> >> cluster don't have, they are preventing other clients from outside of
> >> their cluster that connect to them from being able to get the pieces
> >> from them.
> >>
> >> If all the slaves were operating 100% by the protocol the following
> >> things would happen:
> >> 1> They would continue to download pieces already obtained within their
> >> cluster, either from within their cluster or from outside (I don't care
> >> which personally).
> >> 2> They would try to not preference internally to the exclusion of all
> >> external peers.
> > 
> > This is not required to conform to the BitTorrent protocol. This is
> > merely conventional client design.
> > 
> > Peers will merely download those pieces from other peers that are not
> > part of the cluster. The effect you're concerned with isn't serious until
> > the meta-peer is massively larger than the swarm (until then the peers
> > are very likely to have connections to non-cluster peers), and it isn't
> > useful to employ a meta-peer that large because it cannot utilize its
> > full download nor upload.

> And what if there are only two peers in the entire swarm that have a
> given piece and one of them is in the meta-peer and it isn't sharing
> with the rest of the swarm?

Well, it will be handled by standard swarm dynamics. Peers will see it is
rare, and tend to download it and spread it across the swarm.

Having said that, why wouldn't a meta-peer not share a piece it has? It
wants to download just as much as anyone else, so of course it is going
to upload the piece! Just it will only be uploading the piece from
specific localities within the swarm, mainly only the one or two group
members who have the piece will upload it. The swarm then has to take
care of spreading the piece around. How is this distinct from the case of
any other peer?

Meta-peers are not about *only* uploading to their master. Meta-peers are
about prefering uploading to their master. In order to obtain pieces to
upload to their master though, they have to act mostly like any other
peer, and upload blocks other peers request from them. In order to do
their job, slaves have to mostly act like normal peers. Just will only
ever bother to download a subset of the torrent (mainly pieces that no
other member of the group has downloaded).

I'd tend to keep most of the logic of upload and download dispersed
amoungst the members of the group. Mainly the master only broadcasts the
list of pieces already in progress to the slaves. The slaves get to
choose which pieces to download based on their local view (they have to
have a queue of pieces they're planning to download with the master in
order to avoid problems with RTT, but this is a known technique). This is
simplest and requires the least interactive bandwidth amoung the members.
Plus it will likely provide the best utilization as they will grab pieces
most likely to be requested by their local peers. How are a bunch of
peers with only two extra pieces of logic going to destroy a swarm?

> >> But part of the original proposal had exactly what I mentioned, having
> >> one peer in the cluster redirect the traffic for that piece to another
> >> peer in the cluster.
> > 
> > I didn't read that in what CoolByte was suggesting, nor have I ever
> > deliberately suggested such a thing. The only traffic redirection
> > CoolByte appears to of suggested was that the slaves upload pieces to the
> > master for no compensation (meta-BT, really). Not something a normal peer
> > would do, but still conformant to the BitTorrent protocol.

> CoolByte did recommend the redirection of piece requests in his original
> proposal. It is beyond the capabilities of the protocol.

Well, then I disagree with CoolByte and support your statement on this
specific issue.

> It is the option of the peers to upload to the central system without
> expecting anything in return, however the only members of a swarm that
> should be doing this are seeds.

Sure, normally. I still don't see why providing free uploads to a master
destroys a swarm though.



> >>>> Attacks such as?
> >>> Hordes of zombies can easily cause disruption in a swam, but even without
> >>> those...
> >>>
> >>> One method is to advertise posession of all pieces, but ensure no one
> >>> ever downloads more than one block from you. By corrupting one block of
> >>> every piece, the clients will have to discard 224KB of data merely for
> >>> 32KB of bandwidth on your part. Oh, make that discard 240KB of data for
> >>> merely 16KB of bandwidth, the smaller block makes this attack a lot more
> >>> efficient. Magnifying your effecting corrupting bandwidth by a factor of
> >>> 15 is pretty powerful. The problem here being that you cannot immediately
> >>> verify every block you download.
> > 
> >> I've seen this attack.
> > 
> >> Most implementations of this attack come from a single /16 or smaller
> >> subnet that can be trivially blocked with an ip blocker application.
> >> Additionally, the long-term effectiveness of such an attack against a
> >> given swarm is almost nonexistent due to the automatic banning
> >> algorithms already in place in modern mainstream clients.
> > 
> > I'm surprised they're so braindead as to limit themselves to a single
> > /16. At issue though is if you're talking to an evil peer, how do you
> > confirm that it is evil? You may need to download the entire piece from
> > other peers (and how do you know there isn't an evil one there?). True,
> > the evil peer can be uncovered pretty quickly, but until then it is still
> > causing 16 times its bandwidth to be wasted (upload the one block, the
> > piece needs a full download to confirm evilness of peer). That kind of
> > magnification factor is nasty. Even the more limited factor of 2 waste is
> > pretty decent. I must consider the inability to verify block a definite
> > weakness (made worse by smaller block sizes).

> Most clients don't ban on a single hashfail with a given peer involved
> due to the potentially large number of peers involved with a piece,
> however long-term effects due to the bans on repeated hashfails are
> insignificant at best due to the fact that hashfailed pieces aren't
> shared and peers that repeatedly send bad pieces get banned.

Someone actively trying to destroy the swarm won't care that they can't
download.

> > Though, equally, have you confirmed that a meta-peer will damage a swarm?
> 
> There is only one situation where it would not:
> Conditions:
> 1> all members of the meta-peer have a larger uplink between each other
> than they do to the rest of the swarm and/or have a separate uplink
> between each other and the rest of the swarm

DVDs in the mail fit this.

Uploading to their master first will also be similar to this situation.
I'd tend towards slaves incrementally uploading to their master, rather
than waiting until the end. By giving preference to their master, that
upload will squelch the swarm uplink, guarenteeing that it will
effectively have greater bandwidth than the swarm uplink.

I don't see how this will prevent damage to the swarm, but this condition
is met anyway.

> 2> all members exchange pieces with each other SIMULTANEOUS to sharing
> with the swarm (effectively giving each member peer a public and private
> share pipe, letting member peers share full speed to outside peers and
> high-full speed separately to inside peers).

In order to fulfill their job, the slaves must do this anyway. If they
don't upload to their local peers, they won't download from their local
peers, and therefore they'd be unable to upload to their master (at which
point, why would anyone bother implementing this?). This condition is
met.


This sounds like you're worried about a pool of leeches trying to get
together to multiply the effect of optimistic unchokes. That is something
to worry about, but that is an active attack, not what I am suggesting,
and hopefully not what CoolByte was suggesting elsewhere.


-- 
(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \BS (    |         EHeM at gremlin.m5p.com PGP 8881EF59         |    )   /
  \_CS\   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/





More information about the BitTorrent mailing list