[BitTorrent] Re: Request for protocol extension: get_info/info messages

Elliott Mitchell ehem at m5p.com
Wed Jan 12 19:14:17 EST 2005

>From: iain_wade <iwade at optusnet.com.au>
> --- In BitTorrent at yahoogroups.com, Elliott Mitchell <ehem at m...> wrote:
> > >From: iain_wade <iwade at o...>
> > > The intention of our caching peer is to have it passively
> > > listening for connections solely from our own customer base,
> > > not to participate in the general torrent distribution.
> > >
> > > To be able to do this, we need two things. The first is to get
> > > the clients to connect to us. We need that to happen by default
> > > as relying on user configuration would limit this feature to
> > > a small fraction of the user base. (I have a http proxy (for
> > > tracker communications) written which will add itself to the
> > > list of peers before returning, but getting a few hundred
> > > thousand people to change their settings is going to be a
> > > problem for us).
> >
> > Why not replace the returned list with *just* your cache?
> >
> > The simplest approach might be to return one IP:port pair for
> > each peer the tracker returns. By then imitating the client
> > actions, you can reasonably effectively imitate how the client
> > is acting and correctly tit for tat each peer. Add in one extra
> > record that is purely the cache to provide cached blocks
> > (prevents a peer from getting over-credited).
> Because that would require proxying all communications to the outside
> world, which causes a number of problems (and helps out with a few
> others).


> It also significantly complicates the code required.

Makes it so that care must be taken when writing it, but I must disagree.
Should be roughly the same amount of code.

> Think performance and scalability.

Assign different pieces to a series of hosts, assign different ranges of
IP addresses to be proxyed through different servers.

> We have significantly more broadband customers than the 64k available
> ports, let alone multiplying that by the number of peers they are
> talking to.


A connection is defined by the tuple, source address, source port,
destination address and destination port. You're not even come close to
exhausting that space. You will need multiple connections per port, not
the most commonly used setup, but entirely within the capabilities of any

> As implemented now though, there is an option on the proxy to just
> return itself in which case you will only receive blocks that other
> users have already downloaded and are available in the cache.

Problem is the client will attempt to get more peers if you only give it

> As for implementing a tit-for-tat algorithm for "correctness". This
> cache just slams the data out as fast as it can, like any good cache
> should.

In other words you're caching for the whole world? Great, what is your
proxy's IP address and port? I'd love such an unlimited cache!

I was suggesting that with the scenario I was suggesting you don't need
to implement tit for tat. You only need to proxy the client's actions. If
a client receives a block and disconnects you know the peer is goofy.

> > > I would like the following patch to be integrated into the
> > > official BitTorrent client (as well as any other clients) for
> > > this reason. It does a single dns lookup each time the program
> > > starts for "btcache.p2p" and adds the ip addresses returned to
> > > the peer list:
> > >
> > > https://habitue.net/projects/bt/btcache.patch
> >
> > Problem is this pollutes things for folks without a cache in from
> > of them. Why is this needed? If you can proxy the tracker
> > connection, why is better than modifying the response?
> I don't consider one extra DNS lookup per program start "pollution".

One lookup for an invalid domain. Slowing most systems. This also
strongly advertises your use of BitTorrent to the ISP, who shouldn't be
informed for privacy reasons (they can analyze the traffic to find BT,
but your suggestion provides a clear indicator).

> With almost a million customers and a quarter of a million broadband
> ones, communicating that change becomes too big a deal. For Windows
> XP users running the official client it means going into an obscure
> control panel and adjusting their environment variables which could
> possible affect other software.

Your million customers represent less than 1% of the Internet. Making
a cache easy to utilize is acceptable, making intrusive changes IMO is

> Caching is most effective when more people are using it.
> The best case from our perspective is to get it enabled by default.
> The source to the cache is available for other folks to run as well.
> If it as effective as expected then others may choose to run it.

Thing is there will be plenty who either explicitly do not wish to use
it, or don't have one handy. For them it is a disadvantage.

> > > The second feature would be the get_info/info extension so we can
> > > obtain the "pieces" and "piece length" fields needed for sensible
> > > torrent participation.
> > >
> > > https://habitue.net/projects/bt/btgetinfo.patch
> > 
> > Why do you need to ask the client this information for sensible
> > participation? You can proxy with the piece# plus offset being the
> > keys and everything will fine. Work on 16/32K blocks and it works
> > fine (either you have to join 16K blocks together for clients that
> > use 32K blocks, or split 32K blocks for clients with 16K blocks).
> > If you see the client ask for the piece again, you can guess that
> > you've got bogus data in your cache. 
> >
> > I've suggested the "by_hash" mode to solve a similar situation.
> > The nice part about by_hash is you also add deniability for the
> > cache. You're caching and cannot be forced to reveal or police
> > what your clients are downloading (because you cannot know).
> A man-in-the-middle proxy for all communications is not a scalable
> option for peer to peer networks. One option would be to implement the
> proxy cache only for actual data transfers but that would require much
> larger and more invasive patch and I believe this will be almost as
> effective.

And a non-MitM cache is likely to scale? Seems like they'll both run into
a wall at about the same time.

> I'm not interested in deniability, I want to save bandwidth. I don't
> save logs, so associating a blob of data on our disk with a user is
> not possible.

Good goal. Admirable position. My concern is that less scrupulous folks
may choose to log, or could be forced to log via court order. I'd like it
to be that keeping logs isn't useful because they cannot be made to yield
any information.

> Got I hate this YahooGroups interface :-/

Join the club, though as a mailing list it does mostly work.

(\___(\___(\______          --=> 8-) EHM <=--          ______/)___/)___/)
 \   (    |         EHeM at gremlin.m5p.com PGP 8881EF59         |    )   /
  \_  \   |  _____  -O #include <stddisclaimer.h> O-   _____  |   /  _/
    \___\_|_/82 04 A1 3C C7 B1 37 2A*E3 6E 84 DA 97 4C 40 E6\_|_/___/

Yahoo! Groups Links

<*> To visit your group on the web, go to:

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:

More information about the BitTorrent mailing list