[BitTorrent] suggestion for bitorrent: auto updates of DLs

Brian Dessent brian at dessent.net
Tue Dec 14 22:34:34 EST 2004

barrick75 wrote:

> Basically, the idea is for a BT client to keep track of files it has
> previously downloaded, and to be able to automatically check back with
> the source to see if updates of that file have been released.  If
> there is a newer version, it can download automatically/manually (user
> preference) etc.

I think it would be much more fruitful to try to add some kind of
multi-sourcing to rsync, rather then add the rsync protocol to BT. 
BitTorrent is not suitable at all to downloading changesets.  If you
have a large binary file and make a change to it that affects its
length, then BT is going to redownload the entire thing.  rsync on the
other hand will only send the segments that have changed.  If you make
in-place changes to the file such that its length does not change, then
BT has a chance of working in a rsync-like way, because it would not
need to redownload the segments/slices that did not change.  But this is
very fragile, especially when multiple files are involved.  BT was never
designed for that and so it will not work the way you want it to.

Even if you're talking about just adding new files and not modifying
existing ones, it doesn't really fit in with BT.  The way it works now
is that the list of files is sorted by path/filename and then the hashes
are calculated as if it was one large concatenation of all the
individual files.  Thus if you add a new file to a multifile torrent it
might as well be the same as having one large file and inserting data in
the middle, changing its length.  The hash boundaries would change and
no longer line up, and so you'd end up redownloading the whole entire
thing even if there was only one new file added.  This kind of sucks,
and I think it will be addressed in "v2" when files will be treated
individually, and not taken as one large monolithic chunk of data (for
the purposes of hashing.)

But really, rsync pretty much does what you want already.  It would be
much more suited to maintaining a set of linux packages, or any content
that is incrementally changed.  It doesn't have swarming or anything
like that, of course.

And, the whole "bittorrent for web sites" thing keeps coming up over and
over again, but it's just not a good idea.  It was never designed for
such, and will not work well at all.  BT is really geared towards large
files that do not change and that are expected to take long periods of
time to transfer, not web sites that are small in size and change often.


------------------------ Yahoo! Groups Sponsor --------------------~--> 
Make a clean sweep of pop-up ads. Yahoo! Companion Toolbar.
Now with Pop-Up Blocker. Get it for free!

Yahoo! Groups Links

<*> To visit your group on the web, go to:

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:

More information about the BitTorrent mailing list