[bittorrent] Re: Intersecting Torrent data ???
David P. Mott
dpmott at sep.com
Tue May 10 11:30:07 EDT 2005
Sorry, I'm going to combine a couple of responses in here.
> From: Charles Iliya Krempeaux <supercanadian at gmail.com>
> Subject: [bittorrent] Intersecting Torrent data ???
> Are there any "standard" techniques being used when you want the user to
> download a sequence of files via BitTorrent??? Let me illustrate what I
> am asking with an example....
> So, what are people currently doing in situations like this? Are there any
> techniques for situations like this?
I can tell you what I do, but it's strictly a client-side trick. Let's
say that there are three torrents:
where the first two torrents together equal the content of the last.
Let's also say that the filenames have changed between them, because
someone decided to change filenames on their machine before creating the
last torrent and seeding it.
I can make hardlinks on an NTFS partition (and all of you *nix fans can
make symlinks) such that you only have one copy of the actual data. You
can start all three torrents if you like, however I highly recommend that
you use a client that allows you to set a "do not download" status on each
file in the torrent. Use this feature to make sure that a given file is
getting downloaded from only one swarm (i.e. only one torrent).
In this way, you can download from the fastest swarm (you have to choose
which one it is), and when you're done, you can seed all three from one
set of files (and some cleverly crafted symlinks).
You could choose to make a "super torrent" that includes all possible
files, and then let peers use a bittorrent client that allows them to set
a "do not download" status on some of those files. Then, they can get the
files that they want, and none of the ones that they don't want.
>> BitTorrent is not designed for sequential downloading. So you'd have
>> to download the entire archive before you can play the first file.
> Has anyone created any extension to BItTorrent to do this? (I.e., is there
> already a defacto standard for this?)
There are a couple flavors of "super-seeding", as I recall, one of which
may seed in sequential order. This involves sending out "HAVE" messages
to peers to encourage them to download in a specified order.
However, if their bandwidth is limited, then your HAVE messages may arrive
at a frequency too high for them to turn around and ask you for that
piece, without ignoring (or deferring) some of the HAVE messages.
It's important to understand that you cannot *make* any bittorrent peer
download *anything*, and certainly not in any specific order. It's like
configuring NT -- you don't configure NT, you "make suggestions".
If you want to download in sequential order, then you have to go write
your own bittorrent client (or better, use a protocol and client
specifically designed for that task). You can play with some of the
features in some existing clients, for instance Azureus allows you to set
priority of "high", "low", and "do not download" on each file in the
torrent. So, if you care to baby-sit your download, you can set the first
few files to a "high" priority, and when they get done, set the next few,
and so on.
In the end, however, your ability (as a peer or a tracker) to influence
the behavior of any other given peer in the swarm (or all peers in the
swarm) are very limited and bordering on non-existent.
I think all of that is considered "A Good Thing (tm)", because it allows
each peer to determine how, where, and when to get it's best download
speeds and to follow a pattern of download behavior that is either
dictated by the user (via a GUI or config file) or dictated by embedded
algorithms designed to prolong the life of the swarm. It's my
understanding that sequential downloading is a necessity for streaming
media, but it lends no positive aspects to a high-speed download protocol
whose purpose does not include streaming media.
Or, in short: bittorrent is not, was not, and probably never will be a
streaming protocol. The very traits that make it fast are the same traits
that make it horrible for sequentially streaming media. Sorry.
More information about the BitTorrent