Skip to Content.
Sympa Menu

bluesky - RE: Grapevine - scalability

bluesky AT lists.ibiblio.org

Subject: Global-Scale Distributed Storage Systems

List archive

Chronological Thread  
  • From: Tony Kimball <alk AT pobox.com>
  • To: bluesky AT franklin.oit.unc.edu
  • Subject: RE: Grapevine - scalability
  • Date: Tue, 11 Dec 2001 21:02:06 -0600


Quoth Lucas Gonze on Tuesday, 11 December:
: Yeah, no disagreement that it's a really useful thing, just that it works
with
: the kind of heavy connection churn you get in public nets like Gnutella.
The
: dropout rate has to be less than the time to fill a hole left by a dropout.
So
: either you have to get the hole filling time below average (mean? minimum?
: don't know) node lifespan and or you have to increase the node lifespan.

I misinterpreted. I thought you were questioning the ability of a
consistent-hash based addressing scheme to responsive to evanescent
publications at high transaction volumes. My back-of-the-envelope
calculations lead me to believe that this is not a problem.

Churn in gnutella is due to several factors that wouldn't be present
in most consistent-hash based distributed stores.

1) Search and drop. This is a usability issue. A distributed store
of any kind should place high priority on not interfering with other
use. My motto is "always on, and never a problem." Gnutella servents
tend to fail this ideal by
a) Making a search-and-drop usage easy
b) Generating traffic which does not prefer higher-priority traffic

2) Incompatible implementations. There are plenty. Incompatibilities
take many forms, from network-polluting policies which beg a
death-penalty, to just plain bugs.

3) Finally, although it's not directly related to churn, the nat2nat
problem bites Gnutella big-time. Any practical CAN that doesn't deal
with this effectively is not suited to release to the broader
Internet. This is indirectly related to churn in several ways.
For example, broadband users, who should make up the bulk and
high-value core of the gnet traffic, are heavily NATed, while the
dial-ups that tend not to be NATed also tend to drop more frequently.

But despite these reasons for my sympathy to your views, I'm not
entirely sure that they are warranted, because

1) On the gnet, persistent servents tend to form persistent
connections over time, and form a highly reliable core. A CAN
with gnutella-like churn could use only *qualified* hosts for
persistent data, and use unqualified nodes only for transient
or opportunistic operations, such as swarming, for example.

2) WAN RAID techniques can extend the useful reliability horizon.
FEC to the rescue again.






Archive powered by MHonArc 2.6.24.

Top of Page