[bittorrent] Help or feature request

András Mocsáry amocsy at gmail.com
Sun Oct 3 07:54:37 EDT 2010


Hello!
I want to ask help, about wheter or not the thing I describe below is
awailable, and if not, then pelase treat it as a proposal.


*Problem deffinition:* I want to distribute regularily updated content.

*Description: *I am relasing content using torrent regurally, being it
binary or text data, I often update them. Each time I create a torrent,
upload it to open trackers, and to the project homepage.
People may or may not find them.

Let me pick on of these scenairos:
My content is an open-source software, I'm distributing both the source and
the builds through torrent (as well as from sourceforge).
Everytime I update the source code, I commit my changes to a source version
control system, create a torrent, and publish it.
These torrents need to be made as often as we release our code to the public
SVN.
If we and the testers are at work, it could happen several times a day, each
resulting in a new torrent.
New torrents mean new need to seed, and new peer list.

*Proposal a:* Since most of our updates are minor, the actual data wich
needs to be distributed is usually *99% the same as it was for the previous
torrent.* This is how incremental version control works. Since 2004, approx
60% of the source has been changed, since 2007 it's only around 30%.
Those who still seed the initial relese have 40% of the torrent data
awailable for the latest one!

 1. DHT: with open source software DHT is fine, and for us it' very
wellcome. Basicly peers can get chunks without a tracker. Since our latest
torrents are generally 99% the same as the previous ones, it would be
Ideal*to have a DHT
*extension *to reroute* already existing *chunks from *the swarm of
the *prevoius
torrent*, *to the new one*, immedately* resulting in 99% awailability.*

 2. Without DHT: The tracker (extension) could somehow *reroute *(with some
trick) *chunk request* to the *older torrent seeders*, and peers, so our
devoted* seeders could seed the latest version* *without *the need for them
to *update *their torrent file *several times a day*.

3. Server side (optional): Tracker could recognize newer, updated versions
of the same torrent, through the use of a version number, and a checkout
link to the head torrent, and could auto-download the latest torrent file.
4. Clientside (optional): Some clients could support version controlled
torrents, by the ability to check out previous or newer versions of a
.torrent file using the same version number and head link supported in the
torrent as it is with the tracker, but also could download older versions
from the tracker.
(another option is to let users choose and download the older torrent from
the torrent head link)


This is similar to Similarity Enhanced Transfer, but Chunk similarity is not
only guaranted by the chunk hash, but additionally guaranted by the
versioning control system, since it is the same code-base, with a little
patch each time.

*Proposal b:* Another possibility is a *purely client side* solution. The
client would handle all versions of the torrent as one, and send relevant
chunks to all peers on all torrents.

Then *peers *can safely *add *a* new version of the torrent* *with *the
ability to leverage the* 99% similarity with the older torrent* if he also
loaded older versions of the torrent into his/her client, *while assuring,
that the* resulting *data is the right version* he/she want's to get.

This could be topped by proposal a's 3. and 4.

Actually The two proposals can complement each other, therefore it could
have been one proposal all toghether.

Additional extensions would be to alctuall handle the torrent files with
real version control systems like SVN, and both servers and client's could
svn checkout the torrents on will. But since .torrent files are generally
small, it would not really have significant bennefit.
Therefore we currently store auto-generated .torrents in a directory of it's
own.

Greets,

András Mocsáry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ibiblio.org/pipermail/bittorrent/attachments/20101003/15dab6bc/attachment.html 


More information about the BitTorrent mailing list