Skip to Content.
Sympa Menu

bluesky - Re: Grapevine Technical Overview

bluesky AT lists.ibiblio.org

Subject: Global-Scale Distributed Storage Systems

List archive

Chronological Thread  
  • From: Stephen Blackheath <stephen AT blacksapphire.com>
  • To: "Global-Scale Distributed Storage Systems" <bluesky AT franklin.oit.unc.edu>, Jim McCoy <mccoy AT io.com>
  • Cc: Anthony Jones <ajones AT clear.net.nz>
  • Subject: Re: Grapevine Technical Overview
  • Date: Sun, 12 May 2002 14:27:59 +1200


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(Tony - thanks for those extra notes you posted.)

Jim,

On Sat, 11 May 2002 06:50, Jim McCoy wrote:
> Part of my curiosity about Grapevine is that it seems to be a weird hybrid
> of MojoNation/Mnet and Freenet.

I am pleased that I have aroused your curiosity. The Grapevine is weird
(which is hardly surprising given that we are weird people), but I am not so
sure about the other accusation.

> I am still trying to figure out what
> problems it solves that are not already solved by one or the other system.
> As each of these two parent systems incorporates more features from the
> other as their development continues I can't quite understand what niche
> need this system is supposed to be fulfilling.
>
> To make the
> security/anonymity claims you are making here you need a large pool of
> hosts running the software, but unless there is a significant win over
> other existing systems there seems to be little compelling reason for
> someone to do so. I guess the big question here is this: outside of a cute
> university research project what is the market for such a system? Who
> needs this and why?

I can't make you acknowledge our project as an equal, but I can request that
you treat us with a little respect. You may want to have a look at our
website.

I can say that our goals have a lot more in common with Freenet than with
MojoNation.

> Having gotten that question out of the way I will now pick a bit at some of
> the claims you are making because my morning mocha was delivered late and
> cold and so I am a bit cranky (and I am not going to risk disrupting my
> caffeine supply by bitching at the surly barrista :)

Perhaps you want to think about cutting down a little.

> > We claim that the Grapevine solves the following specific problems:
> > * Scalability
> > * Traffic efficiency
> > * "Karmic debt"
> > * Response time
> > * Robustness
> > * Resistance to the [15]"Slashdot Effect"
> > * Resistance to Denial-Of-Service attacks
> > * Firewalls
> > * Dynamic IP addresses
> > * High throughput
>
> All of these problems are already solved (and implemented and tested) by
> features in MojoNation/Mnet. Given that you are walking a path already
> explored by this system you might really want to take a look at how it
> worked and ask around to see what problems you are going to run into
> here...

I looked about seven months ago, but could find very little technical
information on the MojoNation website. I just looked now, but the website is
down. You can be sure that I will look again.

> > * Search
>
> This is the hard part for decentralized networks and one which I think than
> Freenet work solved better than others. Information about proposed
> solutions to the decentralized search problem would be really appreciated.

We'll do our best to oblige, time-permitting.

> > * Retrieval of data nearby
>
> Freenet did this well, but I think that this represents a fundamental
> conflict with your anonymity claims. If I know what information is held by
> neighbors then that is the first step in attacking either those sites or
> the availability of the information itself. It will be hard to re-create
> the proper balance between privacy/anonymity and pulling from neighbors
> that Freenet accomplished without just copying the Freenet mechanism.

I am not an expert on MojoNation, but I can claim to be at least a partial
expert on Freenet. To the best of my knowledge, Freenet does not solve the
"retrieving information physically nearby" problem. Have a good look at what
we're doing, and I think you'll see that it has its merits over the Freenet
mechanism.

In our context, I can't see how the knowledge of what files the neighbours
have could be an avenue of attack.

> > * Mix-netting
>
> What is the cost of your solution here? Any scheme which claims to provide
> strong anonymity at the packet level and mixnet features will _always_
> suffer from latency problems and an increase in bandwidth costs. This is
> not negotiable. How does this feature not stomp all over your previous
> claims regarding response time and retrieval of data from neighbors?

I agree that latency is affected - I see it as a tradeoff. Without going
into detail, I am not sure I agree that bandwidth is seriously affected. As
I've explained, we sort of kill two birds with one stone.

> > * Plausible deniability
>
> Plausible deniability claims are not a panacea for legal liability.
> Specifically you would be wise to look around at concepts like "attractive
> nuisance" and "vicarious copyright infringement", especially before you
> plan on any trips to the US or Europe :) Seriously, if you are going to
> make any claims about the legal protections your system might offer I would
> suggest that you talk to some good lawyers. If you want I can direct you
> to a few here in the US who have already gone through the learning curve of
> understanding P2P systems because we (MojoNation) had to walk them through
> it ourselves.

My answer may surprise you, and perhaps it gives a clue as to what our
"niche" is:

I am not making claims about legal protections offered by the Grapevine. I
am making claims about plausible deniability. I do not care about the legal
issues you mention, with one exception: I am almost completely certain that
I am in full compliance with New Zealand law in developing this project.

Another way to put this: You can't make an omelette without breaking a few
eggs.

> [...]
>
> > When a node receives a request for a file, then if it does not have
> > that file, it forwards the request on to its neighbour which is
> > "nearest to" that file.
>
> Ouch. Wave goodbye to that fast response time claim you made earlier.
> This conflict between forwarded requests and direct requests for data seems
> to be a fundamental balancing act that P2P data systems have to decide one
> way or the other. Systems that favor privacy/anonymity like Freenet gain
> privacy benefits by forwarding requests at the cost of increasing latency.
> How is it that Grapevine will avoid these costs?

It's a trade-off of security vs. efficiency.

> > Joining the network
> >
> > Nodes are only permitted to learn about their most immediate
> > neighbours. The network protects itself by keeping all nodes ignorant
> > of the IP addresses of any nodes further afield. In order to complete
> > this protection, we also need a way to make sure a node cannot just
> > join the network repeatedly in different locations to eventually find
> > the IP addresses of all nodes.
> >
> > We do this with a strategy called Solve A Hard Problem [...]
>
> How many nodes does the network need have to prevent me from running a
> batch of hosts in parallel solving the hard problems and then mapping out
> the network? Unless the network is rather large I think that you are
> underestimating how hard it will be to map out the connections between
> nodes. If the hard problem only needs to be solved once by a node then I
> bet I could outrun your real new node growth rate with a good cluster of
> PCs doing nothing but creating virtual nodes to map out the mesh.
>
> If the hard problem needs to be re-calculated after a disconnect then how
> much damage could I do to the network by mapping it partially and then
> taking out those nodes I find which are well-connected? If the cost of
> re-balancing/re-connecting the network when key nodes disappear is large
> enough then I can use the SAHP protection mechanism against the network
> itself (e.g. bounce a few nodes out of the mesh and then use the delay in
> their return to map the remainder of the network a little bit faster.)

You're quite right. As I have described it it only offers a certain level of
protection against attack. I never said it was going to be easy. We have
some more elaborate strategies in the pipeline that should improve things
(see the website). Also see Tony's explanation in a previous email.

> > Key management and plausible deniability
>
> [...]
>
> > In order for someone running a node to know what the contents of a
> > file stored on their system is, they have to guess the filename of the
> > file it belongs to, and then look up the file map and see if that CHK
> > is in the file map. Even if they know the entire contents of the file
> > they suspect, this is not sufficient information, due to the CBC mode
> > initialization vector. Hence plausible deniability.
>
> Actually you are just shifting the legal burden from the hosts with the
> data to the person doing the mapping from filename to file map. This was
> the solution that we used for MojoNation/Mnet and while it solves some
> legal problems it creates a centralized point for attack (legal and network
> attacks) which your network is claiming to not have. If the name->map
> function is distributed then hosts that expected to be able to deny
> knowledge of what is on their servers could find themselves taking on a
> hidden legal liability due to the name->map bits that are on their nodes.

I hadn't considered that: a CHK is obviously a CHK, while a name->map mapping
is obviously not, and this goes against plausible deniability. I'm sure it
can't be too hard to think of a solution to this. For your interest, one
thing we are considering doing (as I think Tony explained) is having the most
trusted nodes both 1. concealed from the rest of the network and 2.
performing the name->map mapping.

> The system sounds interesting from a theoretical point of view but there
> seems to be some claims made here that are in conflict with other claims.
> Clearing some of this up would be greatly appreciated.

I hope I've made a start on that in this message.


Steve
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE83dMv7I0ehz47OHERAmXiAJ9+u73NhI2SvRnlRqwbl2SlbaUatACgxsHu
Q3j1PVDr9sdOLpGZeJoFQeI=
=IiBd
-----END PGP SIGNATURE-----




Archive powered by MHonArc 2.6.24.

Top of Page