[BitTorrent] Re: Replacing web content.

jason_b_alonso jalonso at hardwick.hackorp.com
Wed Nov 3 10:42:36 EST 2004


I'm very much interested in seeing a "WebTorrent" implementation. 
However, I believe that we can accomplish WebTorrents with much a
simpler design (iterations follow):

WebTorrents, version 0

How: run a torrent of an entire web site's directory structure.  User
browses local filesystem copy of torrented site.

Problems: Versioning.  This idea came to mind when I saw that
electoral-vote.com was attacked/slashdotted and could not serve its
frequently-updated content.  BitTorrenting a mirror (reasonable, since
the site is "small") would require frequent updates of the torrent file.

WebTorrents, version 1

How: Add versioning (methods addressed shortly).  When a client
contacts a tracker with an outdated version number, the tracker
prompts the client to download an updated torrent file.  Client
responds automatically.  Hash data for already-downloaded files is
compared against the new torrent file, and all is well (differential
downloads!).  What's more, a user can continue to browse the local
mirror (as before), while seeding, and updates should happen
automatically (with proper tracker prompting) while the user is browsing.

Problem 1: How do we accomplish versioning?  (Keep reading).

Problem 2: How does a tracker know if an update is legit?  This wasn't
a problem with static torrents, because a "static" torrent is
self-certifying by nature.

Problem 3: Does a client delete files that disappear?  I would say no,
and declare it a non-problem.

Versioning solution A

Add versioning by appending a number to the filename suffix (i.e.
my-site-mirror.torrent.v15).  No-brainer from there.  This should be
pretty close to backwards/forwards compatible (not there, but close).

Versioning solution B

Add some sort of metadata record to the torrent file and tweak the
protocol to exchange version numbers formally (you should be groaning).

WebTorrents, version 2

How: Add a digital signature (gpg/openssl... something open source) to
the torrent file metadata.  Stick the public key in there too, so any
torrent in a series certifies its successor torrents.  You get some
additional whizbang coolness out of this too, like "yeah, I'm
certifying this torrent."

Well, that's my two cents.  Thoughts?

Jason Alonso

--- In BitTorrent at yahoogroups.com, Brad Neuberg <bkn3 at c...> wrote:
> Olaf, there is a project that is working on what you are interested in 
> called Webtorrent.  I blog a bit about it here:
> At 08:36 AM 10/15/2004, Olaf van der Spek wrote:
> > > TCP/IP has a slow start issues.  Each time you click a link you can
> > > see this.
> >
> >You can?
> >What about persistent HTTP connections?
> >
> > > That is why we have ARP cache/proxy, DNS cache, cached
> >
> >Eh, ARP and DNS are UDP only.
> >
> > > routing tables, web Proxy cache servers, etc.  If you had one
> > > connection to the actual site tracker, and spread that file
> > > out over multiple peers, save the files to cache, then view them
> > > cache - each link would open up much quicker.
> > >
> > > While I agree that one could take the concept of BitTorrent websites
> > > and implement it very poorly, I also think that it could be done
in a
> > > way that would be benifitial.  Especially down the road.
> > >
> > > The reason I posted was to see if there is any RFC or discussion
> > > boards currently on this topic.  While I can imaging taking the
> >
> >There is (AFAIK).
> >There's a mod for a webserver and a mod for Mozilla that implements
> >But I don't know the URL.
> >
> > > current source to make this concept a reality would take a skilled
> > > programmer maybe a month or so, working 8 hours 5 days a week
> > > (estimate)  I would imagine the RFC/concept phase could take
much longer.
> > >
> >
> >
> >Yahoo! Groups Sponsor
> >click here
> >
> >[]
> >
> >
> >
> >----------
> >Yahoo! Groups Links
> >    * To visit your group on the web, go to:
> >    * 
> >

> >
> >    *
> >    * To unsubscribe from this group, send an email to:
> >    * 
> >
<mailto:BitTorrent-unsubscribe at yahoogroups.com?subject=Unsubscribe>BitTorrent-unsubscribe at yahoogroups.com

> >
> >    *
> >    * Your use of Yahoo! Groups is subject to the 
> > <http://docs.yahoo.com/info/terms/>Yahoo! Terms of Service.
> Brad Neuberg, bkn3 at c...
> Senior Software Engineer, Rojo Networks
> Weblog: http://www.codinginparadise.org
> =====================================================================
> Check out Rojo, an RSS and Atom news aggregator that I work on.  Visit 
> http://rojo.com for more info. Feel free to ask me for an invite!
> Rojo is Hiring!  If you're interested in RSS, Weblogs, Social
> Java, Open Source, etc... then come work with us at Rojo.  If you
> someone and we hire them you'll get a free iPod!  See 
> http://www.rojonetworks.com/JobsAtRojo.html. 
> [Non-text portions of this message have been removed]

------------------------ Yahoo! Groups Sponsor --------------------~--> 
$9.95 domain names from Yahoo!. Register anything.

Yahoo! Groups Links

<*> To visit your group on the web, go to:

<*> To unsubscribe from this group, send an email to:
    BitTorrent-unsubscribe at yahoogroups.com

<*> Your use of Yahoo! Groups is subject to:

More information about the BitTorrent mailing list