[bittorrent] Re: Intersecting Torrent data ???

David P. Mott dpmott at sep.com
Tue May 10 11:30:07 EDT 2005

Sorry, I'm going to combine a couple of responses in here.

> From: Charles Iliya Krempeaux <supercanadian at gmail.com>
> Subject: [bittorrent] Intersecting Torrent data ???
> Are there any "standard" techniques being used when you want the user to 
> download a sequence of files via BitTorrent??? Let me illustrate what I 
> am asking with an example....
> So, what are people currently doing in situations like this? Are there any
> techniques for situations like this?

I can tell you what I do, but it's strictly a client-side trick.  Let's 
say that there are three torrents:


where the first two torrents together equal the content of the last. 
Let's also say that the filenames have changed between them, because 
someone decided to change filenames on their machine before creating the 
last torrent and seeding it.

I can make hardlinks on an NTFS partition (and all of you *nix fans can 
make symlinks) such that you only have one copy of the actual data.  You 
can start all three torrents if you like, however I highly recommend that 
you use a client that allows you to set a "do not download" status on each 
file in the torrent.  Use this feature to make sure that a given file is 
getting downloaded from only one swarm (i.e. only one torrent).

In this way, you can download from the fastest swarm (you have to choose 
which one it is), and when you're done, you can seed all three from one 
set of files (and some cleverly crafted symlinks).


You could choose to make a "super torrent" that includes all possible 
files, and then let peers use a bittorrent client that allows them to set 
a "do not download" status on some of those files.  Then, they can get the 
files that they want, and none of the ones that they don't want.

>> BitTorrent is not designed for sequential downloading. So you'd have
>> to download the entire archive before you can play the first file.
> Has anyone created any extension to BItTorrent to do this? (I.e., is there
> already a defacto standard for this?)

There are a couple flavors of "super-seeding", as I recall, one of which 
may seed in sequential order.  This involves sending out "HAVE" messages 
to peers to encourage them to download in a specified order.

However, if their bandwidth is limited, then your HAVE messages may arrive 
at a frequency too high for them to turn around and ask you for that 
piece, without ignoring (or deferring) some of the HAVE messages.

It's important to understand that you cannot *make* any bittorrent peer 
download *anything*, and certainly not in any specific order.  It's like 
configuring NT -- you don't configure NT, you "make suggestions".


If you want to download in sequential order, then you have to go write 
your own bittorrent client (or better, use a protocol and client 
specifically designed for that task).  You can play with some of the 
features in some existing clients, for instance Azureus allows you to set 
priority of "high", "low", and "do not download" on each file in the 
torrent.  So, if you care to baby-sit your download, you can set the first 
few files to a "high" priority, and when they get done, set the next few, 
and so on.

In the end, however, your ability (as a peer or a tracker) to influence 
the behavior of any other given peer in the swarm (or all peers in the 
swarm) are very limited and bordering on non-existent.

I think all of that is considered "A Good Thing (tm)", because it allows 
each peer to determine how, where, and when to get it's best download 
speeds and to follow a pattern of download behavior that is either 
dictated by the user (via a GUI or config file) or dictated by embedded 
algorithms designed to prolong the life of the swarm.  It's my 
understanding that sequential downloading is a necessity for streaming 
media, but it lends no positive aspects to a high-speed download protocol 
whose purpose does not include streaming media.

Or, in short: bittorrent is not, was not, and probably never will be a 
streaming protocol.  The very traits that make it fast are the same traits 
that make it horrible for sequentially streaming media.  Sorry.


More information about the BitTorrent mailing list