[BitTorrent] Re: Request for protocol extension: get_info/info messages

iain_wade iwade at optusnet.com.au
Wed Jan 12 02:57:49 EST 2005

--- In BitTorrent at yahoogroups.com, Elliott Mitchell <ehem at m...> wrote:
> >From: iain_wade <iwade at o...>
> > The intention of our caching peer is to have it passively
> > listening for connections solely from our own customer base,
> > not to participate in the general torrent distribution.
> >
> > To be able to do this, we need two things. The first is to get
> > the clients to connect to us. We need that to happen by default
> > as relying on user configuration would limit this feature to
> > a small fraction of the user base. (I have a http proxy (for
> > tracker communications) written which will add itself to the
> > list of peers before returning, but getting a few hundred
> > thousand people to change their settings is going to be a
> > problem for us).
> Why not replace the returned list with *just* your cache?
> The simplest approach might be to return one IP:port pair for
> each peer the tracker returns. By then imitating the client
> actions, you can reasonably effectively imitate how the client
> is acting and correctly tit for tat each peer. Add in one extra
> record that is purely the cache to provide cached blocks
> (prevents a peer from getting over-credited).

Because that would require proxying all communications to the outside
world, which causes a number of problems (and helps out with a few

It also significantly complicates the code required.

Think performance and scalability.

We have significantly more broadband customers than the 64k available
ports, let alone multiplying that by the number of peers they are
talking to.

As implemented now though, there is an option on the proxy to just
return itself in which case you will only receive blocks that other
users have already downloaded and are available in the cache.

As for implementing a tit-for-tat algorithm for "correctness". This
cache just slams the data out as fast as it can, like any good cache

> > I would like the following patch to be integrated into the
> > official BitTorrent client (as well as any other clients) for
> > this reason. It does a single dns lookup each time the program
> > starts for "btcache.p2p" and adds the ip addresses returned to
> > the peer list:
> >
> > https://habitue.net/projects/bt/btcache.patch
> Problem is this pollutes things for folks without a cache in from
> of them. Why is this needed? If you can proxy the tracker
> connection, why is better than modifying the response?

I don't consider one extra DNS lookup per program start "pollution".

We have a plugin available for Azureus which caches the results for
three days, but I didn't want to complicate this patch and there was
no configuration/storage facility available in the official client.

I can only proxy tracker connections when the user specifically adds
a proxy setting (i.e. set http_proxy environment variable with the
official client, or Azureus has a GUI setting).

With almost a million customers and a quarter of a million broadband
ones, communicating that change becomes too big a deal. For Windows
XP users running the official client it means going into an obscure
control panel and adjusting their environment variables which could
possible affect other software.

Caching is most effective when more people are using it.
The best case from our perspective is to get it enabled by default.

The source to the cache is available for other folks to run as well.
If it as effective as expected then others may choose to run it.

> > The second feature would be the get_info/info extension so we can
> > obtain the "pieces" and "piece length" fields needed for sensible
> > torrent participation.
> >
> > https://habitue.net/projects/bt/btgetinfo.patch
> Why do you need to ask the client this information for sensible
> participation? You can proxy with the piece# plus offset being the
> keys and everything will fine. Work on 16/32K blocks and it works
> fine (either you have to join 16K blocks together for clients that
> use 32K blocks, or split 32K blocks for clients with 16K blocks).
> If you see the client ask for the piece again, you can guess that
> you've got bogus data in your cache. 
> I've suggested the "by_hash" mode to solve a similar situation.
> The nice part about by_hash is you also add deniability for the
> cache. You're caching and cannot be forced to reveal or police
> what your clients are downloading (because you cannot know).

A man-in-the-middle proxy for all communications is not a scalable
option for peer to peer networks. One option would be to implement the
proxy cache only for actual data transfers but that would require much
larger and more invasive patch and I believe this will be almost as

I'm not interested in deniability, I want to save bandwidth. I don't
save logs, so associating a blob of data on our disk with a user is
not possible.

Got I hate this YahooGroups interface :-/


Yahoo! Groups Links

<*> To visit your group on the web, go to:

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:

More information about the BitTorrent mailing list