[bittorrent] Friend-machines-assisted approach to boosting one's BT download
dwknight at depthstrike.com
Mon Jun 12 08:02:56 EDT 2006
Elliott Mitchell wrote:
>> From: Harold Feit <dwknight at depthstrike.com>
>> Elliott Mitchell wrote:
>>>> From: Harold Feit <dwknight at depthstrike.com>
>>>> Elliott Mitchell wrote:
>>>>>> From: Harold Feit <dwknight at depthstrike.com>
>>>>>> Elliott Mitchell wrote:
>>> Yes, how does this damage the peer or swarm?
>>> The non-cluster peer can still download those pieces from another
>>> non-cluster peer. Unless a single meta-peer is at least 40 times a swarm
>>> it joins, I don't see it causing any sort of problems. Crucially there is
>>> little point in employing a meta-peer larger than around half the size of
>>> a swarm because at that point, the meta-peer's bandwidth will likely
>>> outperform the swarm's, at which point additional members won't increase
>>> download speed.
>> With a meta-peer that has not downloaded the piece, that interaction is
>> forced rather than voluntary, causing a negative skew in piece distribution.
If the meta-peer is actively not trying to download pieces already in
the cluster, it won't have those pieces until late in the torrent
progression, forcing the torrent off.
>>> At least with my thoughts, the individual slaves are still act pretty
>>> similarly to other peers. They would still retrieve what appear to be
>>> rarer pieces, since other local peers are likely to want to download
>>> those, in exchange for being able to download other pieces they still
>>> As a /group/ the meta-peer still downloads one copy, and has to upload
>>> roughly one copy. Yes, that copy is broken into distinct sets scattered
>>> across the swarm, but this is nearly an ideal situation for seeding. Any
>>> pieces not held by a local member of the cluster will be obtained via
>>> other peers. The swarm still holds at least two distributed copies, one
>>> on the meta-peer and one across all the peers the meta-peer downloaded it
>>> Where is the damage?
>> When the meta-peer leaves, they take the ENTIRE meta-peer's COMBINED
>> upload rate AND distribution away from the swarm.
>> In smaller swarms, this can be fatal to the piece distribution for the
>> given torrent.
> Yes, what about people on OC12s leaving swarms though? These two are
> pretty comparable, a node with large amounts of bandwidth leaving a swarm
> will be noticable, but not fatal.
We aren't talking about OC12s here though, and if someone's on an OC12,
1> worked alone
2> uploaded 3-10 times what they've downloaded because they feel they
can afford to.
> Small torrents are always in jeopardy. Do you tell someone on an OC12 to
> never leave a swarm?
>>>>> Again, slaves and the master conform to the BitTorrent protocol. If they
>>>>> can harm the swam by conforming to the protocol, then the protocol needs
>>>>> to be fixed. I don't see how they would harm the swarm though.
>>>> If they are selectively not downloading pieces that others in their
>>>> cluster don't have, they are preventing other clients from outside of
>>>> their cluster that connect to them from being able to get the pieces
>>>> from them.
>>>> If all the slaves were operating 100% by the protocol the following
>>>> things would happen:
>>>> 1> They would continue to download pieces already obtained within their
>>>> cluster, either from within their cluster or from outside (I don't care
>>>> which personally).
>>>> 2> They would try to not preference internally to the exclusion of all
>>>> external peers.
>>> This is not required to conform to the BitTorrent protocol. This is
>>> merely conventional client design.
>>> Peers will merely download those pieces from other peers that are not
>>> part of the cluster. The effect you're concerned with isn't serious until
>>> the meta-peer is massively larger than the swarm (until then the peers
>>> are very likely to have connections to non-cluster peers), and it isn't
>>> useful to employ a meta-peer that large because it cannot utilize its
>>> full download nor upload.
>> And what if there are only two peers in the entire swarm that have a
>> given piece and one of them is in the meta-peer and it isn't sharing
>> with the rest of the swarm?
> Well, it will be handled by standard swarm dynamics. Peers will see it is
> rare, and tend to download it and spread it across the swarm.
> Having said that, why wouldn't a meta-peer not share a piece it has? It
> wants to download just as much as anyone else, so of course it is going
> to upload the piece! Just it will only be uploading the piece from
> specific localities within the swarm, mainly only the one or two group
> members who have the piece will upload it. The swarm then has to take
> care of spreading the piece around. How is this distinct from the case of
> any other peer?
Normal peers don't attempt to preference one peer over another because
they are told to by a user or authoritative command peer. They
preference because the peer is playing nice.
> Meta-peers are not about *only* uploading to their master. Meta-peers are
> about prefering uploading to their master. In order to obtain pieces to
> upload to their master though, they have to act mostly like any other
> peer, and upload blocks other peers request from them. In order to do
> their job, slaves have to mostly act like normal peers. Just will only
> ever bother to download a subset of the torrent (mainly pieces that no
> other member of the group has downloaded).
> I'd tend to keep most of the logic of upload and download dispersed
> amoungst the members of the group. Mainly the master only broadcasts the
> list of pieces already in progress to the slaves. The slaves get to
> choose which pieces to download based on their local view (they have to
> have a queue of pieces they're planning to download with the master in
> order to avoid problems with RTT, but this is a known technique). This is
> simplest and requires the least interactive bandwidth amoung the members.
> Plus it will likely provide the best utilization as they will grab pieces
> most likely to be requested by their local peers. How are a bunch of
> peers with only two extra pieces of logic going to destroy a swarm?
>>>> But part of the original proposal had exactly what I mentioned, having
>>>> one peer in the cluster redirect the traffic for that piece to another
>>>> peer in the cluster.
>>> I didn't read that in what CoolByte was suggesting, nor have I ever
>>> deliberately suggested such a thing. The only traffic redirection
>>> CoolByte appears to of suggested was that the slaves upload pieces to the
>>> master for no compensation (meta-BT, really). Not something a normal peer
>>> would do, but still conformant to the BitTorrent protocol.
>> CoolByte did recommend the redirection of piece requests in his original
>> proposal. It is beyond the capabilities of the protocol.
> Well, then I disagree with CoolByte and support your statement on this
> specific issue.
>> It is the option of the peers to upload to the central system without
>> expecting anything in return, however the only members of a swarm that
>> should be doing this are seeds.
> Sure, normally. I still don't see why providing free uploads to a master
> destroys a swarm though.
If the master isn't giving back to the swarm directly.
>>>>>> Attacks such as?
>>>>> Hordes of zombies can easily cause disruption in a swam, but even without
>>>>> One method is to advertise posession of all pieces, but ensure no one
>>>>> ever downloads more than one block from you. By corrupting one block of
>>>>> every piece, the clients will have to discard 224KB of data merely for
>>>>> 32KB of bandwidth on your part. Oh, make that discard 240KB of data for
>>>>> merely 16KB of bandwidth, the smaller block makes this attack a lot more
>>>>> efficient. Magnifying your effecting corrupting bandwidth by a factor of
>>>>> 15 is pretty powerful. The problem here being that you cannot immediately
>>>>> verify every block you download.
>>>> I've seen this attack.
>>>> Most implementations of this attack come from a single /16 or smaller
>>>> subnet that can be trivially blocked with an ip blocker application.
>>>> Additionally, the long-term effectiveness of such an attack against a
>>>> given swarm is almost nonexistent due to the automatic banning
>>>> algorithms already in place in modern mainstream clients.
>>> I'm surprised they're so braindead as to limit themselves to a single
>>> /16. At issue though is if you're talking to an evil peer, how do you
>>> confirm that it is evil? You may need to download the entire piece from
>>> other peers (and how do you know there isn't an evil one there?). True,
>>> the evil peer can be uncovered pretty quickly, but until then it is still
>>> causing 16 times its bandwidth to be wasted (upload the one block, the
>>> piece needs a full download to confirm evilness of peer). That kind of
>>> magnification factor is nasty. Even the more limited factor of 2 waste is
>>> pretty decent. I must consider the inability to verify block a definite
>>> weakness (made worse by smaller block sizes).
>> Most clients don't ban on a single hashfail with a given peer involved
>> due to the potentially large number of peers involved with a piece,
>> however long-term effects due to the bans on repeated hashfails are
>> insignificant at best due to the fact that hashfailed pieces aren't
>> shared and peers that repeatedly send bad pieces get banned.
> Someone actively trying to destroy the swarm won't care that they can't
It's when they can't upload because all of the downloading peers have
banned them for being jerk-offs that they start caring.
The next counterpoint is "What if they change IPs?"
To which the counter-counterpoint is "Then that IP will get banned for
repeated hashfails too."
>>> Though, equally, have you confirmed that a meta-peer will damage a swarm?
>> There is only one situation where it would not:
>> 1> all members of the meta-peer have a larger uplink between each other
>> than they do to the rest of the swarm and/or have a separate uplink
>> between each other and the rest of the swarm
> DVDs in the mail fit this.
For real time exchange of pieces? Really?
> Uploading to their master first will also be similar to this situation.
> I'd tend towards slaves incrementally uploading to their master, rather
> than waiting until the end. By giving preference to their master, that
> upload will squelch the swarm uplink, guarenteeing that it will
> effectively have greater bandwidth than the swarm uplink.
> I don't see how this will prevent damage to the swarm, but this condition
> is met anyway.
>> 2> all members exchange pieces with each other SIMULTANEOUS to sharing
>> with the swarm (effectively giving each member peer a public and private
>> share pipe, letting member peers share full speed to outside peers and
>> high-full speed separately to inside peers).
> In order to fulfill their job, the slaves must do this anyway. If they
> don't upload to their local peers, they won't download from their local
> peers, and therefore they'd be unable to upload to their master (at which
> point, why would anyone bother implementing this?). This condition is
> This sounds like you're worried about a pool of leeches trying to get
> together to multiply the effect of optimistic unchokes. That is something
> to worry about, but that is an active attack, not what I am suggesting,
> and hopefully not what CoolByte was suggesting elsewhere.
Unfortunately, CoolByte was all for suggesting abusive and damaging
means to increase his own download speed.
Ultimately, this proposal isn't going to get much support from me, if
any, simply because it's a downloader-centric proposal in a
More information about the BitTorrent