[bittorrent] Re: Intersecting Torrent data ???
David P. Mott
dpmott at sep.com
Wed May 11 11:08:50 EDT 2005
On Tue, 10 May 2005, Charles Iliya Krempeaux wrote:
>> I can make hardlinks [SNIP]
>> In this way, you can download from the fastest swarm (you have to choose
>> which one it is), and when you're done, you can seed all three from one
>> set of files (and some cleverly crafted symlinks).
> Wouldn't that take alot of intervention from the user? I.e., wouldn't the
> user have to know that the different torrents had the exact same file. If
> BitTorrent "identified" whole files (via sha1, sha256, md5, or something),
> it could be done automatically. (But as I understood it, "pieces" have sha1
> hashes, not entire files. And with the "framing" of the pieces of the files
> being potentially different from torrent to torrent, a list of sha1's
> wouldn't help either. Correct me if I am wrong though.)
Yup, it sure does. And yes, given that the piece size may be different
between the two torrents, none of the SHA1 hashes or anything else in the
metafile will be the same, save for the file size. If you have a general
idea about the content, you can make some educated guesses. If you're
worried that the file may be different despite the same file size, you can
download from one torrent, make a backup of the (partial) file, and then
turn loose the other torrent on the file to see if any of the pieces
validate. (Note here that you'd want to start with the torrent that had
the larger piece size, that way the torrent with the smaller piece size
would find some complete pieces. If you do it the other way, the torrent
with the larger pieces may not find that anything is done).
The BT2 protocol (whenever that comes out, I've long ago stopped holding
my breath) is supposed to provide an identifier (i.e. SHA1 Merkle root
hash) for each file. I don't recall if that is affected by piece size,
though. The nice thing about it is that pieces won't overlap files in the
torrent, so you can download and (perhaps at a later date) verify
individual files from a larger torrent.
>> You could choose to make a "super torrent" that includes all possible
>> files, and then let peers use a bittorrent client that allows them to set
>> a "do not download" status on some of those files. Then, they can get the
>> files that they want, and none of the ones that they don't want.
> This is what I was thinking too. Much like you present your files with FTP
> or HTTP (and you pick and choose what you want to download), you could do
> the same with BitTorrent in one "super torrent" as you call it.
> It would also seem to allow more chances for swarming. Since a user would
> "contribute" to files they downloaded long ago from a "super torrent".
> Everytime they wanted to "download" something new from the "super torrent".
Check out http://wiki.theory.org/BitTorrentSpecification, and read the
section on "Algorithms/Super Seeding". A number of bittorrent clients
You can also peruse the "Bittorrent Wish List" page here:
More information about the BitTorrent