[BitTorrent] Re: Request for protocol extension: get_info/info messages

Iain Wade iwade at optusnet.com.au
Wed Jan 12 19:59:58 EST 2005


> To use get_info/info for .torrent-less downloads, users would already
> need to know where to find a tracker and the info_hash value.

I think Olaf was originally proposing that a new URL be defined which
includes the bare minimum of information (info_hash and a few peers)
needed, the client would then connect to the peers and download the
info body and ask for some more peers and it would grow from there.

I don't intend to do any of that, I just want the clients on my
network to connect to me. I'll then start asking them for pieces and
share them around with other users who are downloading the same thing.

It's not the most efficient method, but it is the smallest change
possible and therefore the most likely to be accepted (I hope).

> Another problem with the get_info/info extension is that the load of the
> info message can be hundreds of KBytes.  That's a big message.  Even if
> you download the message correctly, a user might kill the session early,
> wasting the effort of sending the torrent's info in the first place.

The cache only needs to ask one client, once. The performance benefit
to the user gained from using this cache would far exceed the overhead
incurred from a once off upload of a hundred KB. This burden would
probably be distributed amongst the users over time because the first
person connected to torrentA would likely be a different person to the
first on torrentB.

> In my BT friends extension, I do it differently.  After receiving a
> piece, I compute it's SHA1, and then send a 'piece_info' message to a
> peer that has the piece and supports the extension.  It's load is the
> piece index and my computed SHA1 value for the piece.  The peer replies
> with a 'piece_correct' or 'piece_incorrect' message.  Once I've received
> a 'piece_correct' message, I send my HAVE messages.
> 
> This way, only small messages are sent, and the torrent info is only
> sent as you need it.

The problem here is that it is still susceptible to corruption. You
can't trust the other parties in a p2p download without being able to
verify what they've said. The info blob can be verified by comparing
the hash of it to the info_hash on the connection. This ensures all
clients connected with that info_hash are talking about the same
content.

Also, this extension would be more effective in the short term before
it's adoption is widespread because with the get_info/info extension
only one client needs support for the feature and the client only
needs to be available for a short time to upload the info blob, but
with yours at least one client supporting the feature would need to be
available at all times to support checking.

> Reducing the external network traffic for an ISP will obviously save
> them money, but you have to be careful not to upset either your
> customers or the movie and music industries.

This is being done for the customers. I'm sure they will have nothing
but praise once their download start powering ahead at a few hundred
KB/s as soon as they connect.

I'm not a lawyer and don't care one bit about the sensitivities of the
recording and movie industries. If they choose to sue my company we
would either stop doing what we're doing, or if the legal eagles think
it's winnable we might fight them. Not my problem.

We are running a cache for kazaa and edonkey traffic right now and
other copyright infringing content is cached every second of every day
on our http proxy and email servers. In an ISP, caching this content
is a necessity for the smooth running of the network and unavoidable.

There are already commercial peer-to-peer caching solutions available,
however they are expensive and often need to be added inline to the
core of your network, creating an unacceptable single point of
failure.

I don't want to really discuss the legal aspect, and I think these
changes proposed are small enough and generic enough that it's not
important to.

> I'd suggest thinking of your program as a repeater, rather than a cache.
> I assume that your cache acts just like a normal client: download as
> fast as you can, and once you've got the whole thing start acting as a
> seeder.  This has some real problems.  You'd download all kinds of
> torrents, and probably wind up with lots of terrabytes of data.  The
> small problem is that disk cache is expensive.  The big problem is that

This is solved by removing old and/or unpopular content. I assume a
large number of our users download the same content from the same
sources.

> the music/movie industry will probably have a nice chat with your
> employer about seeding their works.  Another problem is that you're not
> helping the torrent while you're downloading.

The cache is effectively Just Another BitTorrent Client(tm) firewalled
to only accept connections from our users and to not need .torrent
files in advance.

so, once it has one piece that a connected user doesn't have it would
be of benefit.

> A repeater, on the other hand, downloads pieces only when it sees a
> chance to help it's peers.  If no connected peers are interested in the
> pieces you currently have, go get another piece.  Otherwise, just seed
> the pieces you have.  Once you've got a bunch of uninteresting pieces,
> exit the torrent, delete the pieces, and re-enter the torrent.  I've
> prototyped this scheme, and it seems to work quite well for torrents
> that have at least a few downloading peers.  It helps with all of the
> problems listed above.  The down-side is it can't help out in torrents
> with only one downloading peer.  It would be interesting to know what
> percentage of download traffic comes from very active torrents vs fairly
> inactive torrents.

This is certainly an option that I've considered, but it's irrelevent
to the inclusing of the patches sent.

Anyone can write a repeater and use the same mechanism i'm proposing
to have client connect to it.

I've prototyped a different algorithm for the cache software, but
others can write their own implementations if they see benefit.

> Is the idea that this would allow isp's to spoof the btchache.p2p
> domain, and return their own bittorent server's address?  Then anyone
> outside an ISP supporting this would get the lookup error, and thus no
> additional peers?

Yes exactly.

> Couldn't this feature be used to keep track of all user downloads?  I
> know this is already possible for ISPs with basic traffic snooping, but
> I think users might worry about an automatic feature that tells his ISP
> that he's about to download a file set.  I'd like to think that my ISP
> tries hard not to look too closely at my traffic.  It'd feel like a
> small invasion of privacy otherwise.

It couldn't track downloads. It could alert the ISP of the use of a
BitTorrent client, but really I think if the ISP cared they can
already determine this and users should be aware that FastTrack
clients already lookup "cache.p2p" and emule already look up
"edcache.p2p".

> Here's a potential alternative: with the BT friends extension, you're
> client could go to the torrents where peers meet to make friends, and
> only make friends with your customers.  When they need help downloading
> from a torrent, they'll contact you with a request_help message, and you
> can go from there.  They don't need to know that you're working for
> their ISP, and hopefully you wont keep a log of the communication.

I would much prefer the clients to connect directly to me without
having to do the run-around :-)

> > Note// I understand JoltId (http://www.joltid.com) have been
> > contacting bittorrent client authors and paying for them to implement
> > a caching proxy solution locked in to their proprietary PeerCache
> > protocol. I would hope any agreements with them do not preclude the
> > fruits of our own work (open source and freely available) being
> > accepted.
> 
> Scary...  I'm not against a guy making a few bucks, or ISPs saving
> money, but these guys are seriously into spy-ware.

I don't know where you got that impression?

>From my experience with them they are good guys with a solid product
but I don't believe that we need to pay their (rather steep) prices for
a BitTorrent cache which we could implement ourselves in an open-source
manner which benefits everyone.

> IMO, this is a good reason for the open-source community to help solve
> the caching problem for free.  So, here's my two cents on how to make it
> actually happen...
> 
> To get support from users, BT client authors, and Bram, I think you need
> a good reason.  Saving the ISP money sounds good, but let's face it: no
> one cares (unless they're being bribed to care).
> 
> Here's a good reason for average Joe to care: you help him download
> faster.  If you do that, I think the path to acceptance will be much
> easier.

That is an important goal of course, and one I already listed above.

Saving bandwidth for an ISP is not something the authors of BitTorrent
clients and the protocol should disregard however. Having an
ISP-friendly protocol (by adding some support for caching) will help
the users of your software because ISP's would be less likely to limit
it's impact through other means like blocking or traffic shaping.

--Iain


 
Yahoo! Groups Links

<*> To visit your group on the web, go to:
    http://groups.yahoo.com/group/BitTorrent/

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:
    http://docs.yahoo.com/info/terms/
 





More information about the BitTorrent mailing list