[bittorrent] Friend-machines-assisted approach to boosting one's BT download
dwknight at depthstrike.com
Sun Jun 11 21:00:19 EDT 2006
Elliott Mitchell wrote:
>> From: Harold Feit <dwknight at depthstrike.com>
>> Elliott Mitchell wrote:
>>>> From: Harold Feit <dwknight at depthstrike.com>
>>>> Elliott Mitchell wrote:
> Yes, how does this damage the peer or swarm?
> The non-cluster peer can still download those pieces from another
> non-cluster peer. Unless a single meta-peer is at least 40 times a swarm
> it joins, I don't see it causing any sort of problems. Crucially there is
> little point in employing a meta-peer larger than around half the size of
> a swarm because at that point, the meta-peer's bandwidth will likely
> outperform the swarm's, at which point additional members won't increase
> download speed.
With a meta-peer that has not downloaded the piece, that interaction is
forced rather than voluntary, causing a negative skew in piece distribution.
> At least with my thoughts, the individual slaves are still act pretty
> similarly to other peers. They would still retrieve what appear to be
> rarer pieces, since other local peers are likely to want to download
> those, in exchange for being able to download other pieces they still
> As a /group/ the meta-peer still downloads one copy, and has to upload
> roughly one copy. Yes, that copy is broken into distinct sets scattered
> across the swarm, but this is nearly an ideal situation for seeding. Any
> pieces not held by a local member of the cluster will be obtained via
> other peers. The swarm still holds at least two distributed copies, one
> on the meta-peer and one across all the peers the meta-peer downloaded it
> Where is the damage?
When the meta-peer leaves, they take the ENTIRE meta-peer's COMBINED
upload rate AND distribution away from the swarm.
In smaller swarms, this can be fatal to the piece distribution for the
>>> Again, slaves and the master conform to the BitTorrent protocol. If they
>>> can harm the swam by conforming to the protocol, then the protocol needs
>>> to be fixed. I don't see how they would harm the swarm though.
>> If they are selectively not downloading pieces that others in their
>> cluster don't have, they are preventing other clients from outside of
>> their cluster that connect to them from being able to get the pieces
>> from them.
>> If all the slaves were operating 100% by the protocol the following
>> things would happen:
>> 1> They would continue to download pieces already obtained within their
>> cluster, either from within their cluster or from outside (I don't care
>> which personally).
>> 2> They would try to not preference internally to the exclusion of all
>> external peers.
> This is not required to conform to the BitTorrent protocol. This is
> merely conventional client design.
> Peers will merely download those pieces from other peers that are not
> part of the cluster. The effect you're concerned with isn't serious until
> the meta-peer is massively larger than the swarm (until then the peers
> are very likely to have connections to non-cluster peers), and it isn't
> useful to employ a meta-peer that large because it cannot utilize its
> full download nor upload.
And what if there are only two peers in the entire swarm that have a
given piece and one of them is in the meta-peer and it isn't sharing
with the rest of the swarm?
>>>>> If the central system advertises possession of pieces it got from
>>>>> slaves, then you will see all pieces that the helpers download advertised
>>>>> from two semi-distinct peers. All pieces the central system downloads
>>>>> from the swam will only be advertised once. This though should not have a
>>>>> major impact on the balance of the swarm (self-interest on the part of
>>>>> normal peers should easily even this out), though it is a decent reason
>>>>> to forbid the central system from advertising possession of pieces it
>>>>> gets from slaves.
>>>> A request to one peer cannot be fulfilled by another peer that has not
>>>> received the same request. A request to one peer cannot be fulfilled by
>>>> another peer when the second peer isn't connected to the requesting peer.
>>> Yes, your point?
>>> Perhaps change that sentence, instead of "from two semi-distinct peers,
>>> with only one visible download" (the download from slave to master not
>>> being visible to the swarm). Odd behavior if your client is observing
>>> very carefully and makes assumptions about precisely how a peer operates,
>>> but still protocol conformant.
>> But part of the original proposal had exactly what I mentioned, having
>> one peer in the cluster redirect the traffic for that piece to another
>> peer in the cluster.
> I didn't read that in what CoolByte was suggesting, nor have I ever
> deliberately suggested such a thing. The only traffic redirection
> CoolByte appears to of suggested was that the slaves upload pieces to the
> master for no compensation (meta-BT, really). Not something a normal peer
> would do, but still conformant to the BitTorrent protocol.
CoolByte did recommend the redirection of piece requests in his original
proposal. It is beyond the capabilities of the protocol.
It is the option of the peers to upload to the central system without
expecting anything in return, however the only members of a swarm that
should be doing this are seeds.
>>>> Attacks such as?
>>> Hordes of zombies can easily cause disruption in a swam, but even without
>>> One method is to advertise posession of all pieces, but ensure no one
>>> ever downloads more than one block from you. By corrupting one block of
>>> every piece, the clients will have to discard 224KB of data merely for
>>> 32KB of bandwidth on your part. Oh, make that discard 240KB of data for
>>> merely 16KB of bandwidth, the smaller block makes this attack a lot more
>>> efficient. Magnifying your effecting corrupting bandwidth by a factor of
>>> 15 is pretty powerful. The problem here being that you cannot immediately
>>> verify every block you download.
>> I've seen this attack.
>> Most implementations of this attack come from a single /16 or smaller
>> subnet that can be trivially blocked with an ip blocker application.
>> Additionally, the long-term effectiveness of such an attack against a
>> given swarm is almost nonexistent due to the automatic banning
>> algorithms already in place in modern mainstream clients.
> I'm surprised they're so braindead as to limit themselves to a single
> /16. At issue though is if you're talking to an evil peer, how do you
> confirm that it is evil? You may need to download the entire piece from
> other peers (and how do you know there isn't an evil one there?). True,
> the evil peer can be uncovered pretty quickly, but until then it is still
> causing 16 times its bandwidth to be wasted (upload the one block, the
> piece needs a full download to confirm evilness of peer). That kind of
> magnification factor is nasty. Even the more limited factor of 2 waste is
> pretty decent. I must consider the inability to verify block a definite
> weakness (made worse by smaller block sizes).
Most clients don't ban on a single hashfail with a given peer involved
due to the potentially large number of peers involved with a piece,
however long-term effects due to the bans on repeated hashfails are
insignificant at best due to the fact that hashfailed pieces aren't
shared and peers that repeatedly send bad pieces get banned.
>>> Simpler, retrieve random single bytes of random pieces that the peer has.
>>> Pretty well forces random seeks of your disk subsystem (unless you can
>>> store the entire torrent in memory). Kills all caches and just plain
>>> murders performance unless your peer charges extra for small requests and
>>> every time a new piece is loaded into memory.
>>> There is no way to counteract the first without changing the protocol
>>> (thankfully the RIAA, MPAA, et al haven't implemented this attack yet).
>> See above.
> Okay, they're braindead. I suspect they'll learn, eventually (and pigs
> might start flying and they might fix their business model).
>>> The second can be countered by careful design of the clients, but to my
>>> knowledge all current clients will be murdered by this.
>> Have you tested to make sure the second attack is actually viable?
> So far it is merely a thought experiment.
> Though, equally, have you confirmed that a meta-peer will damage a swarm?
There is only one situation where it would not:
1> all members of the meta-peer have a larger uplink between each other
than they do to the rest of the swarm and/or have a separate uplink
between each other and the rest of the swarm
2> all members exchange pieces with each other SIMULTANEOUS to sharing
with the swarm (effectively giving each member peer a public and private
share pipe, letting member peers share full speed to outside peers and
high-full speed separately to inside peers).
More information about the BitTorrent