From weidai@eskimo.com Mon Feb 05 22:31:10 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 05 Feb 2001 22:31:02 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id TAA15818 for ; Mon, 5 Feb 2001 19:30:45 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id TAA29051 for bluesky@franklin.oit.unc.edu; Mon, 5 Feb 2001 19:30:45 -0800 (PST) Date: Mon, 5 Feb 2001 19:30:44 -0800 From: Wei Dai To: bluesky@franklin.oit.unc.edu Subject: test Message-ID: <20010205193043.A28792@eskimo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i This is a test. From hal@finney.org Thu Feb 08 15:34:49 2001 X-Mailer: Lyris Web Interface Date: Thu, 8 Feb 2001 15:34:41 -0500 Subject: Test message To: bluesky@ibiblio.org From: "Hal Finney" Is this on? I am sending from the web interface at http://franklin.oit.unc.edu/cgi-bin/lyris.pl?enter=bluesky but I don't know the address to use for sending email to the list. One of the issues which has been discussed on the Freenet mailing list is the difficulty of engineering a system suitable for use in a country where the government is restricting access to the net. A good example today is China, but of course in the future more countries may have restrictions. Even today, in the U.S. there are potential dangers in running peer to peer software due to the Digital Millennium Copyright Act. This act requires those offering file distribution to publish an address where copyright owners can contact them to request that files be removed, and then of course the operators must comply. There is a conflict between the need to make your system's address known to others in the system (so that files can be shared) and the need to keep it secret from government or private agencies who want to restrict what you read and publish. Hal From weidai@eskimo.com Fri Feb 09 02:14:04 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Feb 2001 02:14:01 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id XAA23253 for ; Thu, 8 Feb 2001 23:13:53 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id XAA11233 for bluesky@franklin.oit.unc.edu; Thu, 8 Feb 2001 23:13:56 -0800 (PST) Date: Thu, 8 Feb 2001 23:13:56 -0800 From: Wei Dai To: bluesky@franklin.oit.unc.edu Subject: Re: Test message Message-ID: <20010208231356.F17756@eskimo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i I've changed the list HELLO message to include the list address. Apparently the email server at ibiblio.org hasn't been set up to forward bluesky@ibiblio.org to bluesky@franklin.oit.unc.edu, so please use the latter address until we get that sorted out. On Thu, Feb 08, 2001 at 03:34:41PM -0500, Hal Finney wrote: > One of the issues which has been discussed on the Freenet mailing list is > the difficulty of engineering a system suitable for use in a country where > the government is restricting access to the net. A good example today is > China, but of course in the future more countries may have restrictions. I believe the solution to this problem is regulatory arbitrage and onion-routing type techniques. If we have a system where the owner of the storage hardware can decide what to store, censored data will migrate to nodes where they are not controlled. For people who are inside the controlled areas, the question is how to communicate with nodes that are blocked by the government firewalls. If the system is popular enough so that the government cannot afford to block every single node that's part of it, people inside can always reach any outside node by connecting to a non-blocked node and tunnel to the target node through it. So basicly we can present the censors with a binary choice: either block the entire system or none of it. You can consider this as an argument for building a general purpose global-scale distributed storage system that happens to be censorship resistant, rather than one that's designed to be censorship resistant at the cost of efficiency. The latter will not become popular enough to make it too expensive to block entirely. From Ted Anderson Fri Feb 09 07:16:20 2001 Message-ID: <3A83DF8A.57674527@transarc.com> Date: Fri, 09 Feb 2001 07:16:10 -0500 From: Ted Anderson Organization: Transarc Corporation X-Mailer: Lyris List Manager Newsgroup Interface MIME-Version: 1.0 Newsgroups: bluesky To: bluesky Subject: Re: Test message References: <93578@bluesky> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit > Is this on? The list is on. The problem at the moment is that you always have to translate ibiblio.org to franklin.oit.unc.edu when sending mail. > I am sending from the web interface[1] but I don't know the address to > use for sending email to the list. The bluesky@franklin.oit.unc.edu address should work. As Hal mentioned, members can post messages using the mailing list maintenance webpage. Posting via NNTP through the newsgroup also works. Mailing list traffic can be viewed by anyone in a web archive[2] and in netnews[3] format. This is a *public* mailing list, though, posting is limited to subscribers. This limitation is to avoid spam, but I support anonymous submissions, and unless the membership is opposed, I'll try to work something out to accomodate such contributors. Let me know. I've set up a temporary webpage[4] for the list, which contains the charter, the invitation you've all seen, and some of this administrivia. I would like to expand the webpage with annotated links to projects related Global-Scale Distributed Storage Systems and other resources for those interested in the topic. Let me know if you have suggestions. I'll leave comments on Hal's technical points to another message. Ted Anderson [1] http://franklin.oit.unc.edu/cgi-bin/lyris.pl?enter=bluesky [2] http://franklin.oit.unc.edu/cgi-bin/lyris.pl?visit=bluesky [3] news://franklin.oit.unc.edu/bluesky [4] http://www.transarc.com/~ota/bluesky From graydon@venge.net Sat Feb 10 20:55:23 2001 Return-Path: Received: from spitfire.velocet.net ([209.167.225.66]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 10 Feb 2001 20:55:15 -0500 Received: from venge.net (H134.C192.tor.velocet.net [216.138.192.134]) by spitfire.velocet.net (Postfix) with SMTP id 072AF19A013 for ; Sat, 10 Feb 2001 20:55:11 -0500 (EST) Received: (qmail 6054 invoked by uid 1000); 11 Feb 2001 01:53:10 -0000 From: graydon@venge.net Date: Sat, 10 Feb 2001 20:53:10 -0500 To: bluesky@franklin.oit.unc.edu Subject: Re: Test message Message-ID: <20010210205310.F22111@venge.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i On Thu, Feb 08, 2001 at 03:34:41PM -0500, Hal Finney wrote: > One of the issues which has been discussed on the Freenet mailing list is > the difficulty of engineering a system suitable for use in a country where > the government is restricting access to the net. A good example today is > China, but of course in the future more countries may have restrictions. imho this is going to be an issue dictated by politics and law -- what the courts will and will not buy wrt plausible deniability. you can go encrypted, plain or steganographic; secret-shared or whole-file; session key or client key. all of it comes down to what the country happens to have outlawed and who happens to be presiding when they haul you into court. eventually (I think the hope is) this stuff will be so ubiquitous that it's _known_ to violate laws restricting speech, and hence you are easily targeted by a special-purpose law ("don't run software X"). the politics will come to a head sooner rather than later if the systems don't try to hide what they're being used for. lets focus on making them useable, fault-tolerant, scalable, and searchable. that's hard enough. -graydon From ota@transarc.com Sun Feb 11 09:59:31 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 11 Feb 2001 09:59:25 -0500 Received: from po2.transarc.com.transarc.com (po2.transarc.ibm.com [9.38.192.6]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id JAA30446 for ; Sun, 11 Feb 2001 09:54:08 -0500 (EST) Received: by po2.transarc.com.transarc.com (8.9.0.Beta3/3.15) id for bluesky@franklin.oit.unc.edu; Sun, 11 Feb 2001 09:59:20 -0500 (EST) Received: via switchmail; Sun, 11 Feb 2001 09:59:20 -0500 (EST) Received: from smithfield via qmail ID ; Sun, 11 Feb 2001 09:59:09 -0500 (EST) Date: Sun, 11 Feb 2001 09:59:08 -0500 (EST) Message-Id: To: bluesky@franklin.oit.unc.edu In-Reply-To: References: Subject: User Anonymity From: Ted Anderson On 8 Feb 2001 15:34:41 EST "Hal Finney" said: > There is a conflict between the need to make your system's address > known to others in the system (so that files can be shared) and the > need to keep it secret from government or private agencies who want to > restrict what you read and publish. The reading and publishing sides of this problem are quite different. On 8 Feb 2001 23:13:56 PST Wei Dai said: > If the system is popular enough so that the government cannot afford > to block every single node ... we can present the censors with a > binary choice: either block the entire system or none of it. > You can consider this as an argument for building a general purpose > global-scale distributed storage system that happens to be censorship > resistant, rather than one that's designed to be censorship resistant > at the cost of efficiency. The latter will not become popular enough > to make it too expensive to block entirely. I agree with Wei Dai. This is the right approach to reader anonymity. An efficient and robust system will be naturally censor-resistant (though not censor-proof); count on its being pervasive and indispensable to prevent any effective limits to what users can read. On the publishing side, regulatory arbitrage is certainly going to be important and it is one reason why a global system will be more robust than a system operating at any smaller scale. A key technical feature that is at least as important is the use of content-hash based identifiers: FreeNet's CHK, OceanStore's GUID and MojoNation's block ids. The use of such identifiers frees the reader from having to trust the storage and transport components. Either they are providing correct data in response to requests or they are broken and the distribution system routes around them. Ultimately, the namespace, which maps names to ids, must be authenticated through a trusted path. Isolating the trust requirement to this one fundamental component of the system will make it easier to understand and verify. For publishing, the use of CHKs means that data can be delivered to the reader, or to a congenial jurisdiction, via any and all means available without risk of compromising the data's integrity or utility to the consumer. Email, CD-ROMs by mail, Tee-shirts, or carrier pigeon all can be used in a pinch to avoid exposing the author's address. Once it arrives, if it has the right hash it is the right data, no questions asked. Ted Anderson From hal@finney.org Wed Feb 14 18:08:21 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 14 Feb 2001 18:08:17 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id PAA08456 for bluesky@franklin.oit.unc.edu; Wed, 14 Feb 2001 15:06:00 -0800 Date: Wed, 14 Feb 2001 15:06:00 -0800 From: hal@finney.org Message-Id: <200102142306.PAA08456@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Developer liability [Note, reply to the list at bluesky@franklin.oit.unc.edu] This recent Wired article, http://www.wired.com/news/politics/0,1283,41784,00.html, mentions a problem which has occasionally been mentioned among developers working on Freenet: Doug Isenberg, editor of gigalaw.com, says that people creating Gnutella-like clients could be at risk. "I don't think that it's clear from the Ninth Circuit's opinion that the defendants in that situation could avoid liability," Isenberg said, referring to creators of such file-trading applications. "I think (the defendants would have) good arguments but I think they're unlikely to hold up under the Ninth Circuit ruling," Isenberg said. "Had the creators of Gnutella not created that particular application -- by not creating it in the first place -- the creators would have been able to block that copyright infringement." Isenberg is talking about the legal theory of vicarious liability, which requires anyone with the right and ability to avoid infringement -- and a direct financial interest -- to take steps to prevent copyright violations from taking place. Jessica Litman, a professor of law at Wayne State University puts it even more bluntly: "Knowing blindness is actionable." The concern is that even if the P2P system is decentralized and doesn't have a core target like Napster.com, the authors of the software could be exposed to liability. The motivation on the part of content owners would be to discourage the development of new P2P software by scaring people away from working on those programs. Obviously the owners aren't going to recover much money from these impoverished programmers, who are often students working in their spare time (at least that's the case with Freenet). Hal From adam@cypherpunks.ai Wed Feb 14 20:25:43 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 14 Feb 2001 20:25:39 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 8004B49; Wed, 14 Feb 2001 21:25:34 -0400 (AST) Date: Wed, 14 Feb 2001 21:25:34 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: Developer liability Message-ID: <20010214212534.A15056@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from hal@finney.org on Wed, Feb 14, 2001 at 03:06:00PM -0800 Return-Path: Sender: adam@cypherpunks.ai A solution would be to publish the software anonymously, and communicates only using pseudonymous or anonymous mail, and anonymous publishing. Pr0duct Cipher, author of Magic Money would be an example of this type of thing. The anonymous publishing system could have it's own software updates published in it's own distributed document space. I think I saw the gnutella binary on gnutella. The compiler boot strapping exercise for pseudonymous publishing systems. Adam On Wed, Feb 14, 2001 at 03:06:00PM -0800, hal@finney.org wrote: > The concern is that even if the P2P system is decentralized and doesn't > have a core target like Napster.com, the authors of the software could > be exposed to liability. The motivation on the part of content owners > would be to discourage the development of new P2P software by scaring > people away from working on those programs. Obviously the owners aren't > going to recover much money from these impoverished programmers, who > are often students working in their spare time (at least that's the case > with Freenet). From hal@finney.org Thu Feb 15 14:33:33 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 14:33:29 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA11030 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 11:31:14 -0800 Date: Thu, 15 Feb 2001 11:31:14 -0800 From: hal@finney.org Message-Id: <200102151931.LAA11030@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Content based addressing [Subject line changed] Ted Anderson wrote: > A key technical feature that is at least as important is the use of > content-hash based identifiers: FreeNet's CHK, OceanStore's GUID and > MojoNation's block ids. The use of such identifiers frees the reader > from having to trust the storage and transport components. Either they > are providing correct data in response to requests or they are broken > and the distribution system routes around them. Freenet uses the content hash as not only an identifier, but as an address for the data. The idea is conceptually simple: each node holds not only data, but pointers to the nodes from which that data came. When data is requested from the net by hash, if a node does not have the data it forwards it to the node which holdes the closest hash. The same forwarding rule is used when data is inserted into the net. The effect is that data with close hashes tends to congregate together on certain nodes, and you can follow a path through the network that leads you to the data you want. No broadcasting is necessary, and you can find data with a limited number of hops. At least, that's what the simulations show. The network in actual use seems to be somewhat fragmented and balky, but there are few people using it and the code is still buggy, so the hope and expectation is that the mature network will work well. Tim Berners-Lee had some relevant comments about new forms of URIs at http://www.w3.org/DesignIssues/Architecture.html: A few spaces are worthy of note which in which identity is fairly well defined, but have no defined dereferencing protocol: the message identifier (mid) and content identifier (cid) spaces adopted from the MIME world, the md5: hash code with verifiable pure identity, and the pseudo-random Universally Unique Identifier (uuid) from the Apollo domain system and followers. These may be underused as URIs. I'm not familiar with all these identifier families but it seems that at least some of them use the same idea, a content based hash as a unique identifier for the document. Freenet, and perhaps the other systems as well, then turns the URI into a URL so that the document can be fetched by hash. Hal From hal@finney.org Thu Feb 15 17:34:32 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 17:34:29 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id OAA11486 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 14:32:14 -0800 Date: Thu, 15 Feb 2001 14:32:14 -0800 From: hal@finney.org Message-Id: <200102152232.OAA11486@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity Ted Anderson, , writes: > On 8 Feb 2001 23:13:56 PST Wei Dai said: > > If the system is popular enough so that the government cannot afford > > to block every single node ... we can present the censors with a > > binary choice: either block the entire system or none of it. > > > You can consider this as an argument for building a general purpose > > global-scale distributed storage system that happens to be censorship > > resistant, rather than one that's designed to be censorship resistant > > at the cost of efficiency. The latter will not become popular enough > > to make it too expensive to block entirely. > > I agree with Wei Dai. This is the right approach to reader anonymity. > An efficient and robust system will be naturally censor-resistant > (though not censor-proof); count on its being pervasive and > indispensable to prevent any effective limits to what users can read. I don't think that jurisdiction shopping can do the job. Napster would probably not have been successful if the only people who could offer music were located in those few countries beyond the reach of the record industry. Most people with internet access are located in Western countries where treaties, international business interests and common cultures will lead to the same data being banned in all of them. Much of the attraction of decentralized data storage comes from the ability to share data that is legally threatened. Otherwise the many free web portals would be adequate for most people's use. Or do readers see P2P as being primarily about improving the efficiency of data sharing beyond what is possible today? Hal From hal@finney.org Thu Feb 15 17:45:28 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 17:45:20 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id OAA11529 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 14:43:05 -0800 Date: Thu, 15 Feb 2001 14:43:05 -0800 From: hal@finney.org Message-Id: <200102152243.OAA11529@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Freenet's hashing algorithm Ted Anderson, , writes: > For publishing, the use of CHKs means that data can be delivered to the > reader, or to a congenial jurisdiction, via any and all means available > without risk of compromising the data's integrity or utility to the > consumer. Email, CD-ROMs by mail, Tee-shirts, or carrier pigeon all can > be used in a pinch to avoid exposing the author's address. Once it > arrives, if it has the right hash it is the right data, no questions > asked. This is one of the big advantages of content based hash schemes. Freenet delivers the data by streaming it back through the chain of nodes that the request followed to find the data. Each node in the back chain caches the data as it passes through. This serves to spread popular data throughout the network, one of the design goals of Freenet. However this mechanism is vulnerable to spoofing. A node which supplies bogus data in response to a request might not have the failure detected until the entire data packet has been received and hashed. This could tie up considerable network resources. In order to avoid this, Freenet adopts a hash based on the paper by Gennaro and Rohatgi, "How to Sign Digital Streams", from Crypto 97. The idea is that you break your data up into blocks, and then create a chained hash. Hash the last block and store the hash with the preceding one; hash that block (including the stored hash) and store that with the next preceding one; and so on, up to the first block. The hash of the first block (with its stored hash of the rest of the message) is the content hash of the message as a whole. With this mechanism, the data can be check for compliance with the hash as each block is received. The worst a misbehaving node can do is to return incomplete data, cutting off before the last block is sent. It also means that a node cannot falsely respond to a request unless it has at least a prefix of the requested data, which reduces the possible impact on the network. I don't know if this kind of hash would be suitable for other networks, but perhaps it would be helpful in some cases. Hal From weidai@eskimo.com Thu Feb 15 21:31:06 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 21:30:56 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id SAA13236 for ; Thu, 15 Feb 2001 18:30:38 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id SAA01906 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 18:30:38 -0800 (PST) Date: Thu, 15 Feb 2001 18:30:38 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: User Anonymity Message-ID: <20010215183038.I11981@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Thu, Feb 15, 2001 at 02:32:14PM -0800 On Thu, Feb 15, 2001 at 02:32:14PM -0800, hal@finney.org wrote: > I don't think that jurisdiction shopping can do the job. Napster would > probably not have been successful if the only people who could offer > music were located in those few countries beyond the reach of the record > industry. Most people with internet access are located in Western > countries where treaties, international business interests and common > cultures will lead to the same data being banned in all of them. I think censorship resistance will ultimately depend on market mechanisms. Suppose each storage server charges money to deliver requested data. As a data object is supressed and becomes rarer, its price goes up and more servers will enter the market to serve it. What the equlibrium price turns out to be depends on what penalties the servers face and the probability of getting caught. Let's talk about how to reduce the penalties (for example through plausible deniability) or the probability of getting caught. To be concrete, let me propose an enforcement mechanism the authorities may use. A list of banned hashes is published. Any server found to be storing or transporting any object that hashes to an entry in the list will be confiscated. Now the enforcers periodicly request items on the list, and confiscates any server that responds. This mechanism seems effective against any conceivable storage system, so perhaps if jurisdiction shopping can't do the job, nothing else can. From weidai@eskimo.com Thu Feb 15 21:45:01 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 21:44:56 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id SAA19900 for ; Thu, 15 Feb 2001 18:44:42 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id SAA02686 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 18:44:42 -0800 (PST) Date: Thu, 15 Feb 2001 18:44:42 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Content based addressing Message-ID: <20010215184442.K11981@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Thu, Feb 15, 2001 at 11:31:14AM -0800 On Thu, Feb 15, 2001 at 11:31:14AM -0800, hal@finney.org wrote: > Freenet uses the content hash as not only an identifier, but as an > address for the data. The idea is conceptually simple: each node holds > not only data, but pointers to the nodes from which that data came. > When data is requested from the net by hash, if a node does not have > the data it forwards it to the node which holdes the closest hash. > The same forwarding rule is used when data is inserted into the net. > The effect is that data with close hashes tends to congregate together > on certain nodes, and you can follow a path through the network that > leads you to the data you want. No broadcasting is necessary, and you > can find data with a limited number of hops. > > At least, that's what the simulations show. The network in actual use > seems to be somewhat fragmented and balky, but there are few people using > it and the code is still buggy, so the hope and expectation is that the > mature network will work well. I think there are fundamental problems with the Freenet content location system. I posted these concerns to freenet-chat, but never got a meaningful response. I'll copy it here: Freenet's search algorithm is characterized as "steepest-ascent hill climbing search with backtracking". I wonder how it deals with landscapes that doesn't have a single hill top. There seem to be two problems. First is multiple hill tops, or multiple nodes that specialize in nearby keys. When you insert a document with a key near these keys, the insert request will be attracted to one of these nodes. But requests for the document may be initially attracted to the other nodes. Is there some provision for quickly going back downhill when the document you're looking for is not at the local hilltop, instead of wasting time searching the slopes of the hill? The second problem is a flat landscape. Suppose you insert a document with a key near the key of a very popular document that is cached everywhere. The insert request and subsequent data requests for this document will then be routed randomly and probably no one will be able to find it. From hal@finney.org Thu Feb 15 22:09:09 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 22:09:04 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id TAA12964 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 19:06:50 -0800 Date: Thu, 15 Feb 2001 19:06:50 -0800 From: hal@finney.org Message-Id: <200102160306.TAA12964@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity [BTW is the bluesky@ibiblio.org address working yet? I have been sending to bluesky@franklin.oit.unc.edu.] Wei writes: > I think censorship resistance will ultimately depend on market mechanisms. > Suppose each storage server charges money to deliver requested data. As a > data object is supressed and becomes rarer, its price goes up and more > servers will enter the market to serve it. What the equlibrium price turns > out to be depends on what penalties the servers face and the probability > of getting caught. Popular P2P systems today are charging nothing for services. Given the absence of micropayment infrastructure, systems which rely on payment face great difficulties. Even the Mojo Nation has backed away from the notion of Mojo as an actual payment medium [1]. Assuming that global file sharing continues to increase in popularity in the next year or two, and that some system becomes dominant, chances are that it will not depend on payments because that is not a prospect in that time frame. > Let's talk about how to reduce the penalties (for example through > plausible deniability) or the probability of getting caught. To be > concrete, let me propose an enforcement mechanism the authorities may use. > A list of banned hashes is published. Any server found to be storing or > transporting any object that hashes to an entry in the list will be > confiscated. Now the enforcers periodicly request items on the list, and > confiscates any server that responds. This mechanism seems effective > against any conceivable storage system, so perhaps if jurisdiction > shopping can't do the job, nothing else can. This could have been used against Napster, but the conventional wisdom is that going after end-users is not cost effective. In practice you will need to observe due process, and ultimately you've got to send some policemen to the guy's house to take the computer. This costs a lot of money per confiscation. And with 20 million Napster users the odds of any particular person getting caught are tiny. I think they tried confiscations against anonymous ftp sites in the pre Napster days but it didn't do much good. Attempting to enforce confiscation for transporting (as composed to storing) verboten data is going to be tricky. You can't confiscate the Internet backbone or AT&T's phone lines just because Metallica MP3s travelled over them. That's the point of the DMCA exemptions. But these legal exemptions can perhaps be exploited by the P2P networks so that the ultimate source of the data is not seen by the end user. Maybe they can craft the laws to distinguish these cases but it is not easy. Also don't forget that those Napster users can vote too (or will be able to in a few years). Make the laws too Draconian and there could be a backlash. Hal [1] http://www.infoanarchy.org/?op=displaystory&sid=2001/1/21/15160/7105 "But the Mojo-payments are really like karma in the network. These are your bonus points, your frequent flyer miles." From dmolnar@hcs.harvard.edu Thu Feb 15 22:26:24 2001 Return-Path: Received: from hcs.harvard.edu ([140.247.73.252]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 22:26:09 -0500 Received: from localhost (dmolnar@localhost) by hcs.harvard.edu (8.9.3/8.9.3) with ESMTP id WAA24417 for ; Thu, 15 Feb 2001 22:26:05 -0500 (EST) Date: Thu, 15 Feb 2001 22:24:42 -0500 (EST) From: dmolnar To: Global-Scale Distributed Storage Systems Subject: Re: Content based addressing In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII ReSent-Date: Thu, 15 Feb 2001 22:26:02 -0500 (EST) ReSent-From: dmolnar ReSent-To: bluesky@franklin.oit.unc.edu ReSent-Subject: Re: Content based addressing ReSent-Message-ID: On Thu, 15 Feb 2001, Wei Dai wrote: > The second problem is a flat landscape. Suppose you insert a document > with a key near the key of a very popular document that is cached > everywhere. The insert request and subsequent data requests for this > document will then be routed randomly and probably no one will be able to > find it. I know of no provisions in the Freenet protocol for dealing with either problem. I haven't studied the protocol much, though. The idea I want to point out is the one used by the OceanStore project at Berkeley (www.oceanstore.cs.berkeley.edu). Queries in OceanStore have two kinds - a quick, prone to error query and a much slower, but nearly guaranteed to work query. The quick query is a randomized hill-climbing scheme. The claim seems to be that the slow queries always succeed, while the quick queries almost always succeed. Then if pathologies of the kind you describe are rare in practice, the system runs quickly. While it's still really, really important to characterize which search keys lead to pathology in a "quick" scheme and avoid them, maybe having a backup scheme makes it less critical in practice? Thanks, -David From graydon@venge.net Thu Feb 15 23:26:17 2001 Return-Path: Received: from spitfire.velocet.net ([209.167.225.66]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 23:26:13 -0500 Received: from venge.net (H247.C193.tor.velocet.net [216.138.193.247]) by spitfire.velocet.net (Postfix) with SMTP id CD1FA19A013 for ; Thu, 15 Feb 2001 23:26:03 -0500 (EST) Received: (qmail 5995 invoked by uid 1000); 16 Feb 2001 04:22:33 -0000 Date: Thu, 15 Feb 2001 23:22:33 -0500 From: graydon@venge.net To: bluesky@franklin.oit.unc.edu Subject: Re: Freenet's hashing algorithm Message-ID: <20010215232233.B5069@venge.net> Reply-To: graydon@pobox.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i On Thu, Feb 15, 2001 at 02:43:05PM -0800, hal@finney.org wrote: > Ted Anderson, , writes: > > For publishing, the use of CHKs means that data can be delivered to the > > reader, or to a congenial jurisdiction, via any and all means available > > without risk of compromising the data's integrity or utility to the > > consumer. Email, CD-ROMs by mail, Tee-shirts, or carrier pigeon all can > > This is one of the big advantages of content based hash schemes. > this strikes me as a conflation of matters, with serious drawbacks. * lack of friendly names * inability to change data without renaming/moving it * no actual improvement in security. the fact that the data satisfies the hash is vaccuous; any data will satisfy a CHK pointing to it, regardless of who put it there or how "true" it is. what you care about is that the data you retrieved is the data some party intended you to receive, and if you read a CHK off a tee shirt there is no reason to believe this CHK is any more trustworthy than a press release from the government of EvilLand. certainly a good hash algorithm is useful for stretching a friendly name into some fixed-size, uniformly distributed keyspace for finding the associated block. but hashing the content is a security issue, not a retrieval issue. freenet seems to be fiddling around with a variety of key types to get the balance of pragmatics right. chained hashes signed by some sort of pre-existing security association key makes sense for verifying a stream piecemeal, but I still wouldn't use that for locating, in case you want to update the data. you really don't want to relocate all the blocks just because you change 1 byte somewhere. -graydon From hal@finney.org Thu Feb 15 23:31:57 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 15 Feb 2001 23:31:49 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id UAA13129 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 20:29:35 -0800 Date: Thu, 15 Feb 2001 20:29:35 -0800 From: hal@finney.org Message-Id: <200102160429.UAA13129@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Content based addressing Wei writes: > I think there are fundamental problems with the Freenet content > location system. I posted these concerns to freenet-chat, but never got > a meaningful response. I'll copy it here: > > Freenet's search algorithm is characterized as "steepest-ascent hill > climbing search with backtracking". I wonder how it deals with landscapes > that doesn't have a single hill top. There seem to be two problems. Let me briefly describe the Freenet search algorithm for the benefit of other readers (if there are any). More detail is available at http://freenet.sourceforge.net/index.php?page=icsi-revised. Each document is identified by a key, typically a 160 bit hash. Each node stores a set of documents which have been sent to it and requested for storage. Along with the document it of course stores the key, but also the address of another node which has or had the document. (Over time, documents may be expunged to save space, but the key and node address are preserved for a longer period to help with finding the document on another node.) The search algorithm is simple. When the node receives a request for a given key, if it has the corresponding document it returns it. If not, it looks through its data store for the closest key to the requested one, and forwards the request to the corresponding node. Then, when the data streams back (if it is found) the node saves a copy and sets the node pointer to the node from which the data came. The same routing algorithm is used for insertion as for data requests. When the data is inserted it is stored on this node and then the data store is searched for the closest key, and the data is forwarded to that node to be stored there as well. Both requests and inserts have "hops to live" (HTL) values which tell how far the forwarding should go, and are expected to be roughly logarithmic in the size of the network. The effect of these two algorithms is that nodes will tend to acquire data which is close together in hash. Nodes which hold documents with similar hashes will be likely to point to each other. The network will self-organize into nodes with a degree of specialization (in hash value). Popular data will spread out over more nodes. > First is multiple hill tops, or multiple nodes that specialize in nearby > keys. When you insert a document with a key near these keys, the insert > request will be attracted to one of these nodes. But requests for the > document may be initially attracted to the other nodes. Is there some > provision for quickly going back downhill when the document you're > looking for is not at the local hilltop, instead of wasting time > searching the slopes of the hill? No, there is no such provision. What happens is that if a request is forwarded to the nearest "uphill" node, it may send back a rejection saying that it has already seen that request (and in fact is still waiting for a response from when it forwarded it). In that case the node falls back to the second-best key, and so on. This will tend to move us around the slopes and not jump over to another hill. However I think the hill-climbing metaphor is not that accurate for Freenet. What is the height? It is how close the node is to the desired key. This means that there is not just one fitness landscape, there is a virtually infinite set of them, one for every possible key. Furthermore the distance between nodes in this model depends on what connections they have to each other with respect to keys close to the target, which may also be different for different keys. Therefore as we move through the key space, the network's metric and height will change dramatically. No simple picture of hills and valleys will capture the essence of the Freenet data distribution. The network is also likely to be high dimensional, with each node having a number of connections to other nodes with close keys. This increases connectivity and helps hill climbing to work better. My feeling is that the dynamic process of inserting and fetching documents, combined with the variation in the shape and topology of the fitness landscape with different keys, will facilitate connectivity and help different parts of the landscape to be informed about each other. If there is a split in the network at a certain key value, looking at it with a different key value will see those regions as being connected. Activity on the keys intermediate between the two will help bridge the two parts. Ultimately we need either a mathematical analysis or more detailed simulation results to know whether segmentation becomes a problem in practice. > The second problem is a flat landscape. Suppose you insert a document > with a key near the key of a very popular document that is cached > everywhere. The insert request and subsequent data requests for this > document will then be routed randomly and probably no one will be able to > find it. This could be a problem, but it is an extreme case. If there are other documents in the network closer to the inserted one than the widespread document, eventually the insertion path will divert towards the other documents, after thrashing around randomly for a while. It may be necessary to have a large HTL value for the insertion to work well in this case. If this does become a problem, it might be possible to introduce some new routing rules to break the symmetry. This is done for now in the initial state, which is conceptually perfectly flat. To fix this, initial nodes get random values assigned to them so as to promote self-sorting behavior from the beginning. Perhaps when a single document is found everywhere and we are just following it around, some similar rule could be added. Hal From ota@transarc.com Fri Feb 16 00:14:04 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 00:13:55 -0500 Received: from po2.transarc.com.transarc.com (po2.transarc.ibm.com [9.38.192.6]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id AAA12510; Fri, 16 Feb 2001 00:08:28 -0500 (EST) Received: by po2.transarc.com.transarc.com (8.9.0.Beta3/3.15) id ; Fri, 16 Feb 2001 00:13:45 -0500 (EST) Received: via switchmail; Fri, 16 Feb 2001 00:13:44 -0500 (EST) Received: from smithfield via qmail ID ; Fri, 16 Feb 2001 00:13:40 -0500 (EST) Date: Fri, 16 Feb 2001 00:13:40 -0500 (EST) Message-Id: To: bluesky@franklin.oit.unc.edu, "Global-Scale Distributed Storage Systems" Subject: Re: Developer liability CC: hal@finney.org In-Reply-To: References: From: Ted Anderson On 14 Feb 2001 15:06:00 PST hal@finney.org wrote: > The concern is that even if the P2P system is decentralized and > doesn't have a core target like Napster.com, the authors of the > software could be exposed to liability. The motivation on the part of > content owners would be to discourage the development of new P2P > software by scaring people away from working on those programs. Centralized file systems have servers that can be be subpoenaed or seized when it stores objectional content. The idea that this architecture is necessary for the software to be legal to write is very hard to accept. It seems a bit like the key escrow business. The only encryption software legal (to export) needed a method of recording keys so they could later be recovered by court order. It is my impression that the publication of the powerful "The Risks of Key Recovery..." paper[1] by Abelson et al, put these ridiculous schemes to rest. On the face of it, the idea that certain types of file system software would be illegal to *write*, seems far more ridiculous than that unescrowed encryption would be illegal to export. Perhaps a strong paper outlining the advantages of P2P storage systems would have the same effect. On 15 Feb 2001 14:32:14 PST hal@finney.org wrote: > Or do readers see P2P as being primarily about improving the > efficiency of data sharing beyond what is possible today? Indeed I do. The sort of system I envision is mostly motivated by the desire to have users protect their own privacy and make the best use of their storage resources: servers are just a bottleneck limiting overall system throughput. Swapping copyrighted music is just an incidental, though outlandishly popular, application. Ted [1] http://www.cdt.org/crypto/risks98/ From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 00:55:51 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 00:55:44 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id AAA01113; Fri, 16 Feb 2001 00:57:37 -0500 Date: Fri, 16 Feb 2001 00:57:37 -0500 Message-Id: <200102160557.AAA01113@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: weidai@eskimo.com Cc: bluesky@franklin.oit.unc.edu Subject: dealing with the lack of ecash (Re: User Anonymity) Man this is a nice list, discussions on my favorite topic, cool people, active discussion etc. Like cypherpunks from maybe 5 or more years ago. Some of the same people too. I'd like to thank Ted and Wei for taking the initiative. There was for a brief time some interesting discussion on eternity list about USENET eternity and Anderson's eternity, and distributed file systems in general, but it died out after a while. Wei writes: > I think censorship resistance will ultimately depend on market > mechanisms. Suppose each storage server charges money to deliver > requested data. As a data object is supressed and becomes rarer, its > price goes up and more servers will enter the market to serve > it. What the equlibrium price turns out to be depends on what > penalties the servers face and the probability of getting caught. Economic pressures with real money do help a lot of problems, because arguments like "so use the profits to buy more bandwidth" or "put your price up because demand is too high" are robust answers to DoS and resource allocation. I discussed a bit in the past the idea of using hashcash (or perhaps Wei's b-money, though that I think is a little harder to deploy) as a substitute for cash as no one seems able to deploy ecash. I also proposed a kind of amortizable hashcash for distributed file systems specifically. The idea is basically that you use hashcash, except that each server keeps and generally forwards around the largest collision it has seen for each file it serves. The collision is computed on the filename. (Hashcash will give you a collision longer than you asked for 50% of the time). The "amortization" function for these tokens is just max(x,y), so it's a very crude estimate. There were a few other tricks like protecting against overspending where someone would try to artificially inflate the value of the collision by computing larger collisions in a light-weight security setting presuming you trust servers a little more than users. Anyway the amortizable hashcash idea was to have a lightweight security distributed document popularity metering system which might be one way to arbitrate which documents deserve to be mirrored, which could combat or add resistance against some of the DoS attacks you tend to be able to do against systems that allow free publishing. Also hashcash is kind of pointing out that there is actually some network currency: CPU power, and bandwidth. (Bandwidth is involved because the inference is that in paying with a hashcash token a client has consumed both CPU and network bandwidth to download the content). So while it's not exactly paying because you're just proving what resources you've consumed, so the recipient can't use it directly, he may be able to use it indirectly if he also obtains hashcash voting that his server is a contributor of bandwidth. And so you have this kind of closed system where people are having to contribute to participate a bit like MojoNation's payment system. Perhaps even you'd end up with after markets trading unused server reptuation tokens to allow access. With paypal and other peer-to-peer (not anonymous, but fairly unstoppable and anyway indirect) payment systems after markets are now easier than they were in the days of the DigiCash beta-bucks trial. Adam Personal comments, of course. From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 00:59:41 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 00:59:33 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id BAA01125; Fri, 16 Feb 2001 01:01:27 -0500 Date: Fri, 16 Feb 2001 01:01:27 -0500 Message-Id: <200102160601.BAA01125@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: weidai@eskimo.com Cc: bluesky@franklin.oit.unc.edu Subject: "hot potato" sub-protocol for plausible deniability (Re: User Anonymity) Wei also wrote: > Let's talk about how to reduce the penalties (for example through > plausible deniability) or the probability of getting caught. To be > concrete, let me propose an enforcement mechanism the authorities > may use. A list of banned hashes is published. Any server found to > be storing or transporting any object that hashes to an entry in the > list will be confiscated. > > Now the enforcers periodicly request items on the list, and > confiscates any server that responds. This mechanism seems effective > against any conceivable storage system, so perhaps if jurisdiction > shopping can't do the job, nothing else can. So here's a simple though probably quite inefficient algorithm to have plausible deniability. Each node answers a request just once, then it deletes the content. Now if the attacker discovers some content, the server operator running unmodified software can quite reasonably claim he did not know his server carried that content. If he had used the browsing software to obtain the content, the server would by definition not have the content. Clearly you also need some mechanism to ensure distribution, otherwise content just dies out quickly. One way is to move the data around, like a hot potato, rather than delete it, though this seems likely to be bandwidth inefficient. Graydon proposes something related in his Symptomatic File System [1]: | The motivating cases above are difficult to address primarily | because any illigitimate user wishing to disrupt file service need | only masquerade as a legitimate user long enough to locate the | server with a file they dislike, and then commence an attack on that | server (uniquely determined by IP address). SFS solves this scenario | in an unorthodox way: rather than trying to resist attack, when a | file server S even suspects it is under any form of attack it | immediately falls over, dead as a doornail. | | [...] | | all file servers simply follow an algorithmic rule of thumb: In | order for S to even begin providing file service for a given file | named N, S must first secure a backup location somewhere along the | hash chain for N, and only once the backup acknowledges that it has | a copy can S begin to service requests. Adam [1] http://www.venge.net/graydon/SFS.html From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 01:27:01 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 01:26:58 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id BAA01138; Fri, 16 Feb 2001 01:28:55 -0500 Date: Fri, 16 Feb 2001 01:28:55 -0500 Message-Id: <200102160628.BAA01138@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: weidai@eskimo.com Cc: bluesky@franklin.oit.unc.edu Subject: deployment vectors Wei wrote: > [you could aim to design systems so that...] we can present the > censors with a binary choice: either block the entire system or none > of it. > > You can consider this as an argument for building a general purpose > global-scale distributed storage system that happens to be > censorship resistant, rather than one that's designed to be > censorship resistant at the cost of efficiency. The latter will not > become popular enough to make it too expensive to block entirely. I was thinking about this before in earlier discussion on cypherpunks and eternity. This argument leads me to the conclusion that one should also examine plans which involve very light weight security if even any, and just build systems with other apparent intended properties, with carefully engineered censorship resistant properties tied into them in hard to remove ways. (Think lack of SMTP authentication in early SMTP protocols -- that simple mistake allowed all the unwanted mail problems, and weakened notions of the reliability of mail authorship -- arguably a boon for anonymity in email). So what pressing problems could one fix as a deployment vector to get censor resistance: a good global flash-crowds and DDoS resistance system which has some free decentralised peering system that makes it essentially anarchic and uncontrolable as a virtue in it's method to protect against flash-crowds. (My argument is "you need distributed service to prevent against distributed attacks" and "ultimately you can't distinguish between popular content and a DDoS"). I'd also argue that it's not directly necessary to include reader anonymity in the network, there are systems which do this separately, ZKS freedom, anonymizer, etc. Although it may not be a bad thing to try to do, and certainly making anonymity more widely deployed, available and accessible to the masses is a good thing. My only caution would be to avoid biting off too many hard problems at once. Censor resistant content distribution is a hard enough problem by itself without dragging in anonymous interactive communications which has it's own overlapping but different complexities. A distributed web space able to cope with flash-crowds seems like a fairly killer app. Akamai try to provide high speed, some managed monitoring, etc. but it's still one companies managed system. Something isn't distributed and anarchic which we need for a couple of reasons (censor resistance, and to get a large enough pool of resources for flash-crowds) until it's run by everyone in a preferably zero barrier to entry system, so anyone with resources can join, and users can use, and ISPs have some reason to use it (eg. it reduces load for the same reason as web caches). Perhaps just a next generation mirroring system which uses a reasonable automatic data migration and some simple DNS or otherwise based pragmatic easy solution to lookup to convert names into locations. DNS aggressive caching makes it's hierarchical nature less of a problem for performance, though reserving the right to quickly react to change reduces the cachine opportunities, and hierarchies are bad for censor resistance. Adam From hal@finney.org Fri Feb 16 01:34:31 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 01:34:26 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id WAA13366 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 22:32:11 -0800 Date: Thu, 15 Feb 2001 22:32:11 -0800 From: hal@finney.org Message-Id: <200102160632.WAA13366@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Freenet's hashing algorithm graydon@venge.net writes: > this strikes me as a conflation of matters, with serious drawbacks. > > * lack of friendly names > * inability to change data without renaming/moving it > * no actual improvement in security. the fact that the data satisfies > the hash is vaccuous; any data will satisfy a CHK pointing to it, > regardless of who put it there or how "true" it is. what you care about > is that the data you retrieved is the data some party intended you to > receive, and if you read a CHK off a tee shirt there is no reason to > believe this CHK is any more trustworthy than a press release from the > government of EvilLand. A content hash should be thought of as a shorthand or abbreviation for the document it represents. It is an unforgeable short-name, and in principle an unforgeable address. Whether via Freenet or via some more conventional lookup system, you can have a system to go from hash to data. There are good uses for such documents. Not all documents need to be updated; sometimes you want documents which *can't* be updated: legal agreements, for example. Any time you want to make sure that someone sees a particular revisions of a document you can give them the content hash as a pointer. Archival or historical data can be well served with such hashes. Even when you need updates, content hash addressing can serve as a layer over which you can build an update/revision system, as Freenet is doing. Hal From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 02:14:58 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 02:14:54 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id CAA01216; Fri, 16 Feb 2001 02:16:35 -0500 Date: Fri, 16 Feb 2001 02:16:35 -0500 Message-Id: <200102160716.CAA01216@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: graydon@venge.net Cc: bluesky@franklin.oit.unc.edu Subject: self-authenticating URLs (Re: Freenet's hashing algorithm) It seems there are a number of approaches to URLs. There are arguments for making them readable, persistent across versions, not persistent across versions, self authenticating and so on. Some of the systems with not directly readable URLs are presuming there will be a name resolution server on top. (eg. TAZ/rewebber) (Making URLs self authenticating in a way which requires URLs to change with versions creates it's own problems with hyperlinked docs -- the publius docs describe some of these problems -- they had to go through some complex path to even change one doc because of this, as all references to it had to change, and they had to deal with loops). I read the use in FreeNet to be just an attempt to cheaply authenticate the URL, as the URL is tied to the document. I'd guess the authentication token in FreeNet was re-used as the lookup key just to make URLs small. (If I recall dimly I think I may have suggested using the hash of the file as the URL on the FreeNet list back at the beginning of FreeNets prototyping phase). If someone sends you a URL in a signed PGP mail, or refers to a URL in another document, you may want to know what version of the document they were thinking of. Probably you also want to be able to get to the current version, perhaps you even want the current version authenticated as being by the same author to be the default and just be able to go back to the old version, or navigate around in old versions to avoid revisionism and attempts to silence an author by having him change a later version to say something against his will. So my only argument in this is it's another requirement to be able to get at old versions to avoid revisionism. That may conflict with somone's perceived right to revise information they once made available. However I'm not sure the author can reasonably demand that machines acting in common interest allow him to revise something he once said. I think one should analyse what properties the system should offer, what choices the user will want and try to arrange those properties. Anderson et al have a paper on the Eternal Resource Locator (see above URL for ref) which is interesting about authenticating URLs and hypertext links. My little link farm on these systems includes publius, ERLs and all the rest. http://www.cypherspace.org/links.html Adam > this strikes me as a conflation of matters, with serious drawbacks. > > * lack of friendly names > * inability to change data without renaming/moving it > * no actual improvement in security. the fact that the data satisfies > the hash is vaccuous; any data will satisfy a CHK pointing to it, > regardless of who put it there or how "true" it is. what you care > about is that the data you retrieved is the data some party intended > you to receive, and if you read a CHK off a tee shirt there is no > reason to believe this CHK is any more trustworthy than a press > release from the government of EvilLand. > > certainly a good hash algorithm is useful for stretching a friendly > name into some fixed-size, uniformly distributed keyspace for finding > the associated block. but hashing the content is a security issue, not > a retrieval issue. freenet seems to be fiddling around with a variety > of key types to get the balance of pragmatics right. chained hashes > signed by some sort of pre-existing security association key makes > sense for verifying a stream piecemeal, but I still wouldn't use that > for locating, in case you want to update the data. you really don't > want to relocate all the blocks just because you change 1 byte > somewhere. From weidai@eskimo.com Fri Feb 16 02:23:19 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 02:23:15 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id XAA04274 for ; Thu, 15 Feb 2001 23:23:08 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id XAA15306 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 23:23:08 -0800 (PST) Date: Thu, 15 Feb 2001 23:23:08 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: User Anonymity Message-ID: <20010215232307.A13542@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Thu, Feb 15, 2001 at 07:06:50PM -0800 On Thu, Feb 15, 2001 at 07:06:50PM -0800, hal@finney.org wrote: > [BTW is the bluesky@ibiblio.org address working yet? I have been sending > to bluesky@franklin.oit.unc.edu.] No it's still not working. I've changed the list options so that it doesn't mention ibiblio.org anymore. > Popular P2P systems today are charging nothing for services. Given the > absence of micropayment infrastructure, systems which rely on payment > face great difficulties. Even the Mojo Nation has backed away from the > notion of Mojo as an actual payment medium [1]. Assuming that global > file sharing continues to increase in popularity in the next year or > two, and that some system becomes dominant, chances are that it will > not depend on payments because that is not a prospect in that time frame. I think Mojo Nation's approach is a good one. There is no need to tie Mojo in with real money in the beginning. Just using it as karma is good enough to solve many problems. Eventually if the system becomes popular enough people will naturally start to buy and sell Mojo on their own. > This could have been used against Napster, but the conventional wisdom > is that going after end-users is not cost effective. In practice you > will need to observe due process, and ultimately you've got to send some > policemen to the guy's house to take the computer. This costs a lot > of money per confiscation. And with 20 million Napster users the odds > of any particular person getting caught are tiny. I think they tried > confiscations against anonymous ftp sites in the pre Napster days but > it didn't do much good. Couple of points here. In the Napster case it's obviously much more cost-effective to go after the central server instead of end-users. But here we're talking about decentralized systems so going after end-users will be the only choice. About anonymous FTP sites, I think those tend to be either private (invitation only) or on someone else's server so going after them would be difficult or ineffective. > Attempting to enforce confiscation for transporting (as composed to > storing) verboten data is going to be tricky. You can't confiscate > the Internet backbone or AT&T's phone lines just because Metallica MP3s > travelled over them. That's the point of the DMCA exemptions. But these > legal exemptions can perhaps be exploited by the P2P networks so that the > ultimate source of the data is not seen by the end user. Maybe they can > craft the laws to distinguish these cases but it is not easy. I think it is possible to craft the laws to distinguish between these cases. It just has to say that in case the ultimate source of the data is not known, the last known source is responsible. > Also don't > forget that those Napster users can vote too (or will be able to in a > few years). Make the laws too Draconian and there could be a backlash. My point is that if the authorities were willing to be Draconian enough, then jurisdiction shopping is the only defense available. Is there some intermediate threat level where it would make sense to try to hide the physical location of the server? From hal@finney.org Fri Feb 16 02:45:30 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 02:45:26 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id XAA13619 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 23:43:12 -0800 Date: Thu, 15 Feb 2001 23:43:12 -0800 From: hal@finney.org Message-Id: <200102160743.XAA13619@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity Wei writes: > My point is that if the authorities were willing to be Draconian enough, > then jurisdiction shopping is the only defense available. Is there > some intermediate threat level where it would make sense to try to hide > the physical location of the server? The first part sounds right, but if they are Draconian enough then even jurisdiction shopping will not be enough. They can require monitors in everyone's computers to see what pages they are surfing, etc. The question, as you say, is what is a plausible threat model. For the second part, I'd say that the US, today, under the DMCA fits pretty well. This is from http://www.eff.org/ip/DMCA/hr2281_dmca_law_19981020_pl105-304.html: `Sec. 512. Limitations on liability relating to material online `(a) TRANSITORY DIGITAL NETWORK COMMUNICATIONS- A service provider shall not be liable for monetary relief, or, except as provided in subsection (j), for injunctive or other equitable relief, for infringement of copyright by reason of the provider's transmitting, routing, or providing connections for, material through a system or network controlled or operated by or for the service provider, or by reason of the intermediate and transient storage of that material in the course of such transmitting, routing, or providing connections, if-- `(1) the transmission of the material was initiated by or at the direction of a person other than the service provider; `(2) the transmission, routing, provision of connections, or storage is carried out through an automatic technical process without selection of the material by the service provider; `(3) the service provider does not select the recipients of the material except as an automatic response to the request of another person; `(4) no copy of the material made by the service provider in the course of such intermediate or transient storage is maintained on the system or network in a manner ordinarily accessible to anyone other than anticipated recipients, and no such copy is maintained on the system or network in a manner ordinarily accessible to such anticipated recipients for a longer period than is reasonably necessary for the transmission, routing, or provision of connections; and `(5) the material is transmitted through the system or network without modification of its content. Under this law, you are OK if you are just passing the data through. Servers OTOH are required to remove data upon request of the copyright holder. Hide the server and there is no suppression possible. Hal From weidai@eskimo.com Fri Feb 16 02:46:35 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 02:46:29 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id XAA08941 for ; Thu, 15 Feb 2001 23:46:20 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id XAA16072 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 23:46:24 -0800 (PST) Date: Thu, 15 Feb 2001 23:46:23 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: "hot potato" sub-protocol for plausible deniability (Re: User Anonymity) Message-ID: <20010215234623.B13542@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from adam@cypherspace.org on Fri, Feb 16, 2001 at 01:01:27AM -0500 On Fri, Feb 16, 2001 at 01:01:27AM -0500, Adam Back wrote: > So here's a simple though probably quite inefficient algorithm to have > plausible deniability. Each node answers a request just once, then it > deletes the content. Now if the attacker discovers some content, the > server operator running unmodified software can quite reasonably claim > he did not know his server carried that content. If he had used the > browsing software to obtain the content, the server would by > definition not have the content. I don't think that really accomplishes plausible deniability. I think the criteria for it should be: 1. The server has no way of knowing that it is storing or transporting a banned object or part of a banned object. 2. There is a plausible reason why it can not know; it can't just be the result of a mechanism designed solely to avoid this knowledge. Without (1), the authorities can impose censorship duty on all servers and punish those that don't comply. Without (2), the authorities can ban usage of the special "ignorance" mechanism. From hal@finney.org Fri Feb 16 03:00:52 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 03:00:49 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id XAA13672 for bluesky@franklin.oit.unc.edu; Thu, 15 Feb 2001 23:58:35 -0800 Date: Thu, 15 Feb 2001 23:58:35 -0800 From: hal@finney.org Message-Id: <200102160758.XAA13672@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: deployment vectors Adam Back writes: > So what pressing problems could one fix as a deployment vector to get > censor resistance: a good global flash-crowds and DDoS resistance > system which has some free decentralised peering system that makes it > essentially anarchic and uncontrolable as a virtue in it's method to > protect against flash-crowds. > ... > A distributed web space able to cope with flash-crowds seems like a > fairly killer app. But how do you get censorship resistance from this? Under the DMCA if you are notified by a content owner that you have a certain document that he owns, you must remove it. Streamline this a bit and you have Wei's proposal for a global index of forbidden documents which everyone must check against. I don't see that a system which spreads popular data over multiple servers would avoid this threat. > Akamai try to provide high speed, some managed > monitoring, etc. but it's still one companies managed system. > Something isn't distributed and anarchic which we need for a couple of > reasons (censor resistance, and to get a large enough pool of > resources for flash-crowds) until it's run by everyone in a preferably > zero barrier to entry system, so anyone with resources can join, and > users can use, and ISPs have some reason to use it (eg. it reduces > load for the same reason as web caches). A P2P commercial system with some of these properties is the Swarmcast from OpenCola, http://www.opencola.com/products/4_swarmcast/technology.shtml. "When a user requests a large file from a content provider's site, the user's Swarmcast application automatically locates and downloads chunks of the file from the nearest, least busy computers that have previously downloaded the file. The user's computer downloads simultaneously from as many nodes in the network as possible, maximizing their available download capacity..." Hal From weidai@eskimo.com Fri Feb 16 03:09:45 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 03:09:35 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id AAA15415 for ; Fri, 16 Feb 2001 00:09:29 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id AAA16595 for bluesky@franklin.oit.unc.edu; Fri, 16 Feb 2001 00:09:28 -0800 (PST) Date: Fri, 16 Feb 2001 00:09:28 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: User Anonymity Message-ID: <20010216000928.C13542@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Thu, Feb 15, 2001 at 11:43:12PM -0800 On Thu, Feb 15, 2001 at 11:43:12PM -0800, hal@finney.org wrote: > Under this law, you are OK if you are just passing the data through. > Servers OTOH are required to remove data upon request of the copyright > holder. Hide the server and there is no suppression possible. But if hiding the ultimate server becomes popular, surely the laws will be changed to hold the last known server responsible. I think the reason that's not already the case is not that voters think it would be too draconian, but that the law-makers simply haven't considered the possibility of hiding the server yet. However if we do want to hide the server in order to take advantage of the DMCA exemption, I think a good way to do it would be through rendezvous servers. The ultimate server connects to a rendezvous server through an anonymous communications channel. The rendezvous server then advertises everything stored by the ultimate server as its own, and passes requests and data between the ultimate server and clients. This way we don't impose costs on parts of the system that don't require anonymity, and we don't have to change the system if the DMCA loophole is closed and server anonymity is no longer useful. From weidai@eskimo.com Fri Feb 16 04:17:19 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 04:17:12 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id BAA32308 for ; Fri, 16 Feb 2001 01:17:02 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id BAA17983 for bluesky@franklin.oit.unc.edu; Fri, 16 Feb 2001 01:17:02 -0800 (PST) Date: Fri, 16 Feb 2001 01:17:02 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: the three-services model Message-ID: <20010216011701.D13542@eskimo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i I have been assuming a storage system design that decomposes into three services: 1. name lookup service: maping user-friendly names/URLs to content-hash based ids 2. data location service: maping content-hash based ids to storage servers (network addresses and data transport protocols) that can deliver the data 3. data transport service: actually obtaining content from a storage server given its address and a content-hash based id This results in two levels of indirection. Some current systems seem to have only one level, either a. combining (1) and (2), so that a name is mapped directly to a storage server, or b. combining (2) and (3), so that storage servers are addressed by content-hash base ids. Having an extra level of indirection allows problems with both (a) and (b) to be avoided. The problems with (a) are that the storage server has to be trusted, and that there is waste if the same data object goes by several names in different namespaces and is stored on multiple servers. The problems with (b) are that if a data object changes, it would have to be relocated to a different server, and that servers have little flexibility in choosing what to store. Are there proposed designs that do not fall under the three-services model? From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 08:37:21 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 08:37:18 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id IAA01549; Fri, 16 Feb 2001 08:39:17 -0500 Date: Fri, 16 Feb 2001 08:39:17 -0500 Message-Id: <200102161339.IAA01549@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: weidai@eskimo.com Cc: bluesky@franklin.oit.unc.edu Subject: Re: "hot potato" sub-protocol for plausible deniability (Re: User Anonymity) > On Fri, Feb 16, 2001 at 01:01:27AM -0500, Adam Back wrote: > > So here's a simple though probably quite inefficient algorithm to > > have plausible deniability. Each node answers a request just once, > > then it deletes the content. Now if the attacker discovers some > > content, the server operator running unmodified software can quite > > reasonably claim he did not know his server carried that content. > > If he had used the browsing software to obtain the content, the > > server would by definition not have the content. > > I don't think that really accomplishes plausible deniability. I > think the criteria for it should be: > > 1. The server has no way of knowing that it is storing or transporting > a banned object or part of a banned object. > > 2. There is a plausible reason why it can not know; it can't just be > the result of a mechanism designed solely to avoid this knowledge. > > Without (1), the authorities can impose censorship duty on all servers > and punish those that don't comply. Without (2), the authorities can > ban usage of the special "ignorance" mechanism. You are right with (1). I was restating a much earlier discussion, which had context I omitted. The context was the content is encrypted so that the server can't read it. The document key is also encrypted so the server can't read it. A simple scheme is the one I used in USENET Eternity: both the URL and the document key are derived from the human readable URL with a Key Derivation Function. In this way the only way for the server to discover what it has is to find all eternity URLs and brute force to discover which ones it has. With the addition that the server only serves each file once, this means that absent software modification, or digging in internals the server wouldn't no longer have a block even if it found something using the browsing interface. I added a (3) which has different aims than pluasible deniability: (3) Once an attacker has discovered the server could have served that block, he has destroyed it's ability to further serve the block, and is fairly confident that it never served the block to anyone other than the attacker. I'm thinking that this will make it difficult to prosecute: he didn't serve the block to other people, plus (1) that he didn't know what he was serving. I'm less sure about (2) here you want the purpose of the design construct to have pluasible other purposes than reducing liability. This seems like an odd mix of legal interpretations and design criteria. I also had in mind (4) which was also unstated: (4) The document lookup key should change each time it moves. This makes it very hard to find things, and means you need a distributed name resolution service which is rapidly updated. So this defers the censor ship problem (if you expect all these plausible deniability tricks to hold up in a court) to the distributed name resolution service. The attacker will try to remove the name which is after all static. But a name you would think is less controversial to host, as it's not the data, it's a pointer to the data. Internic typically doesn't get in trouble when a site hosts controversial data. Then again people linking to URLs have been told to remove URLs. However courts have no sense of fair play it seems; ultimately they could just declare the whole thing a RICO conspiracy or something and seize equipment in a draconian way until people are scared off. But I'm not sure such draconian measures are politically achievable. People post all sorts of content on USENET, and there are fights about it but USENET lives. Adam From adam@modemcable005.191-202-24.mtl.mc.videotron.ca Fri Feb 16 09:27:15 2001 Return-Path: Received: from modemcable005.191-202-24.mtl.mc.videotron.ca ([24.202.191.5]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 09:27:06 -0500 Received: (from adam@localhost) by modemcable005.191-202-24.mtl.mc.videotron.ca (8.9.3/8.9.3) id JAA01570; Fri, 16 Feb 2001 09:29:00 -0500 Date: Fri, 16 Feb 2001 09:29:00 -0500 Message-Id: <200102161429.JAA01570@modemcable005.191-202-24.mtl.mc.videotron.ca> X-Authentication-Warning: modemcable005.191-202-24.mtl.mc.videotron.ca: adam set sender to adam@modemcable005.191-202-24.mtl.mc.videotron.ca using -f From: Adam Back To: hal@finney.org Cc: bluesky@franklin.oit.unc.edu Subject: Re: deployment vectors Hal writes: > Adam Back wrote: > > A distributed web space able to cope with flash-crowds seems like a > > fairly killer app. > > But how do you get censorship resistance from this? Under the DMCA > if you are notified by a content owner that you have a certain > document that he owns, you must remove it. Streamline this a bit > and you have Wei's proposal for a global index of forbidden > documents which everyone must check against. I don't see that a > system which spreads popular data over multiple servers would avoid > this threat. That is certainly technically possible, I'm less sure about politically possible. Looking at existing systems for predictors: I'm not sure if we can read the fact that USENET doesn't have this streamlined global content block list as due to the fact that USENET is transient anyway (except for dejanews et al), or due to the fact that USENET doesn't have persistent URLs so it's not so in-your-face to motivate the censors to build such a block list. But either way, users can always make minor changes to the document and republish it. This would lead to some kind of race as the identification is human and manual, and may even involve a court determination, and the service and distribution is lots of comptuers acting very quickly. So this is the kind of pragmatic censor resistance property I was thinking about. You try to engineer an arms race in which the censor loses ultimately, and can never more than inconvenience you. It may also be worth building at least two levels into the system: the convenient fast but fairly vulnerable to censors system, and the slow less convenient system designed to be more ultimately resilient. This would discourage censors as removing the content from the fast system would have limited practical value. Adam From hal@finney.org Fri Feb 16 11:37:42 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 11:37:38 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id IAA14482 for bluesky@franklin.oit.unc.edu; Fri, 16 Feb 2001 08:35:24 -0800 Date: Fri, 16 Feb 2001 08:35:24 -0800 From: hal@finney.org Message-Id: <200102161635.IAA14482@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: deployment vectors Adam writes: > But either way, users can always make minor changes to the document > and republish it. This would lead to some kind of race as the > identification is human and manual, and may even involve a court > determination, and the service and distribution is lots of comptuers > acting very quickly. The problem with this, and with the idea of publishing the content encrypted so the servers don't know what it is (as Freenet does) is that for the system to be useful there must be a public way for users to find out what the new hash/address of the document is. And this way must be relatively automated and widespread, otherwise you're never going to get widespread access to the data (by which I mean millions of people being able to access the file). Sure, you could leak the secret code on IRC to a few dozen other people, but there are plenty of systems in existence today which let you share files secretly on that small scale. The challenge is to come up with something that can be as large as Napster or, ultimately, the web, sharing files just as easily. Any system for widespread publication can be used by the censors as easily as by the users. If we respond to bans by republishing data with minor changes or a new encryption key, then once it goes into the index under the same "friendly name" as the already-banned version, the censor bots will send out the new ban message. Again, you could argue that there might be a small window of opportunity there, a few hours before the file has to be changed again, but I can't see this as being the foundation for a widely used, stable, large-scale file sharing service. With the DMCA there is no court determination initially; the presumption is that the copyright holder is correct, and the server must comply with the withdrawal order. Later, if it turns out that the content owner was wrong, he can get in trouble in court. But the law as written favors content owners. So they can respond very quickly. I don't think the law says how fast the server must respond; they are not required to set up an automated file-removal service and can rely on human intervention. There is also a provision in DMCA that links to bad data are infringing as well. This means that the address lookup service, where we start with a friendly name like a song title and get the latest address on the net, can be forced to remove the entry for the infringing song. This will further hamper attempts to republish data and still somehow get the new address out to the world. This issue arises to some extent with the Napster injunction. The appeals court noted that Napster's server does not have access to the song data. That means that the record companies can't just publish a hash. What Napster has is song titles. But these are user described and have a number of variations, with artist, album, track number and song title sometimes present in various orders, sometimes with misspellings. Can the record companies lay down a blanket ban and say, "you have to remove any song with the word 'Metallica' in it, in any of the following spelling variations?" Probably not. Instead my guess is that they'll use a bot to search the Napster DB for all songs by all of their artists and send Napster a list of all of the variations they found, probably hundreds of thousands of entries. If Napster were to choose to continue to operate and alter their software to comply with such orders, users could respond by a strategy of resubmitting songs under altered names. Perhaps an informal convention could arise and be spread by the grapevine of some specific alterations, like removing the last letter of the artist's name, or adding "RIAA SUX" in the middle of the word. Then the record industry would catch on and send a new ban list to Napster, and we'd have the kind of race you describe. Then we could see how well it works. BTW this suggests another strategy with regard to Napster: if the RIAA casts such a wide net, they will undoubtedly get some false matches - songs which have some of the same words as their artists and titles, but are actually completely different. Someone could compose a tone poem homage to the Virgin Mary and call it "Madonna as a Virgin", and if the RIAA claimed this as one of their songs they would be violating the DMCA and in for penalties. Hal From graydon@venge.net Fri Feb 16 19:34:31 2001 Return-Path: Received: from spitfire.velocet.net ([209.167.225.66]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 19:27:11 -0500 Received: from venge.net (H247.C193.tor.velocet.net [216.138.193.247]) by spitfire.velocet.net (Postfix) with SMTP id B689819A010 for ; Fri, 16 Feb 2001 19:27:01 -0500 (EST) Received: (qmail 6974 invoked by uid 1000); 17 Feb 2001 00:23:25 -0000 From: graydon@venge.net Date: Fri, 16 Feb 2001 19:23:25 -0500 To: Global-Scale Distributed Storage Systems Subject: Re: Freenet's hashing algorithm Message-ID: <20010216192325.E5069@venge.net> Reply-To: graydon@pobox.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i In-Reply-To: ; from hal@finney.org on Thu, Feb 15, 2001 at 10:32:11PM -0800 On Thu, Feb 15, 2001 at 10:32:11PM -0800, hal@finney.org wrote: > A content hash should be thought of as a shorthand or abbreviation for > the document it represents. It is an unforgeable short-name, and in > principle an unforgeable address. Whether via Freenet or via some more > conventional lookup system, you can have a system to go from hash to data. yeah, I get the general idea, but I think you're seriously misapplying the concept of a hash as a trust device. a hash of a chunk of data tells you nothing about the trustworthyness of the chunk. you need a trust relationship with a signing key, and a signature, and a simple hash doesn't give you this. if I get a CHK from a tee-shirt, I've no reason to believe the data I get at that CHK is trustworthy. what or whom I trust is my own business, to be established by verifying signatures against keys I personally acquire. to paint the orthogonality of these matters more concretely: suppose, for instance, that EvilCorp put up a page saying "we are totally legitimate people". I would not trust that, because I am a suspicious guy and have heared some bad stuff about them before. I would know the page was put up by EvilCorp because their signature checks out, but I would not _trust_ what they are saying. if someone then broke the EvilCorp security system, or indeed broke the complete freenet security system, hit every machine storing copies, updated the page with officially (wrt EvilCorp) "wrong" data about the evils of EvilCorp, and then signed the "wrong" page with a key from some activist organization I have a trust relationship with, the page has actually become _more_ trustworthy to me than it was before it was doctored. > There are good uses for such documents. Not all documents need to be > updated; sometimes you want documents which *can't* be updated: legal > agreements, for example. this seems to always come up when discussing distributed storage systems. I think the image of someone "changing" or "censoring" your speech overwhelms the technical fact of the matter: it is exactly the same to have a flag you can set on a document saying "don't accept any updates for me" as it is to have a "non-changeable" document. the two concepts are functionally identical: * any conformant server will reject attempts to change the document * any wrong, evil server will just screw around with the data anyway * any signature-type information, whether used to "locate" the data or simply validate it as it's retrieved, will be broken by an evil, editing server, so the unauthorized edit is detectable by any client I mean, imagine you have "non-changeable" documents. I get one, I put it on my disk, I open a hex editor (if the source is nonfree) and edit the file. I recompute a CHK, I spam the whole universe with the new CHK, posting it on CNN, getting it printed on coke cans, etc. until everyone thinks my CHK is the original. in what way is this any different from someone illicitly updating a document in a system which "allows" updates? it takes roughly as much effort as I'd need to put in to defeat the trust relationship someone has with a signing key they consider relatively "authoritative" anyways. in other words, what does it even mean to "allow" vs. "disallow" updates? imho it's nothing more than a recommendation anyways. compliance is always up to the retriever, with the trust relationship and signature, to ensure. -graydon From hal@finney.org Fri Feb 16 21:07:58 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 16 Feb 2001 21:06:10 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id SAA15731 for bluesky@franklin.oit.unc.edu; Fri, 16 Feb 2001 18:03:57 -0800 Date: Fri, 16 Feb 2001 18:03:57 -0800 From: hal@finney.org Message-Id: <200102170203.SAA15731@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Freenet's hashing algorithm graydon@venge.net writes: > yeah, I get the general idea, but I think you're seriously misapplying > the concept of a hash as a trust device. a hash of a chunk of data tells > you nothing about the trustworthyness of the chunk. you need a trust > relationship with a signing key, and a signature, and a simple hash > doesn't give you this. A CHK is trustworthy in only one respect: you can trust that the returned document is the one pointed to by the CHK. Certainly you can identify other forms of trust that may involve bringing in cryptography and such. But this one simple trust property is still useful. Many naming systems lack this guarantee - URLs on the web today, for example. Hal From hal@finney.org Sat Feb 17 01:08:30 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 01:08:12 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id WAA16183 for bluesky@franklin.oit.unc.edu; Fri, 16 Feb 2001 22:05:59 -0800 Date: Fri, 16 Feb 2001 22:05:59 -0800 From: hal@finney.org Message-Id: <200102170605.WAA16183@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity Wei writes: > However if we do want to hide the server in order to take advantage of the > DMCA exemption, I think a good way to do it would be through rendezvous > servers. The ultimate server connects to a rendezvous server through an > anonymous communications channel. The rendezvous server then advertises > everything stored by the ultimate server as its own, and passes requests > and data between the ultimate server and clients. That seems like a good idea, in general. I think you mean it as an add-on to a P2P system rather than something built into the P2P protocol itself. However if we imagine implementing this for something like Napster, there would be problems. Almost all users are sharing data that they could get in trouble for. So there would have to be rendezvous servers with enough bandwidth to mirror all the data for everyone. It would be hard to create millions of these. I think you'd want to design the system so that everyone was a rendezvous server for someone else. In this way finding a rendezvous peer would be done in an automated way as part of the P2P system. I think building it into the P2P will be necessary to make it transparent and widely useful. > This way we don't impose costs on parts of the system that don't require > anonymity, and we don't have to change the system if the DMCA loophole is > closed and server anonymity is no longer useful. That would be good, but I don't see that it will work to build the rendezvous system as a completely separate product from the P2P. You can't require people to download and install two separate programs. Maybe you could design the system internally so that the rendezvous part was logically separate, but bundle it together with the core P2P application. Then users wouldn't have to be aware of the internal structure. And if at some point it became legal to share data, they could turn off the use of rendezvous hiding. Hal From hopwood@zetnet.co.uk Sat Feb 17 12:16:43 2001 Return-Path: Received: from zetnet.co.uk ([194.247.47.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 12:16:39 -0500 Received: from zetnet.co.uk (man-a144.dialup.zetnet.co.uk [194.247.44.144]) by zetnet.co.uk (8.11.2/8.11.2/Debian 8.11.2-1) with ESMTP id f1HHGXK31244 for ; Sat, 17 Feb 2001 17:16:33 GMT Message-ID: <3A8EB1D8.B3279364@zetnet.co.uk> Date: Sat, 17 Feb 2001 17:16:08 +0000 From: David Hopwood Reply-To: hopwood@zetnet.co.uk X-Mailer: Mozilla 4.7 [en] (WinNT; I) X-Accept-Language: en-GB,en,fr-FR,fr,de-DE,de,ru MIME-Version: 1.0 To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity References: Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: quoted-printable -----BEGIN PGP SIGNED MESSAGE----- Wei Dai wrote: > Let's talk about how to reduce the penalties (for example through > plausible deniability) or the probability of getting caught. To be > concrete, let me propose an enforcement mechanism the authorities may u= se. > A list of banned hashes is published. Any server found to be storing or= > transporting any object that hashes to an entry in the list will be > confiscated. Now the enforcers periodicly request items on the list, an= d > confiscates any server that responds. This mechanism seems effective > against any conceivable storage system, so perhaps if jurisdiction > shopping can't do the job, nothing else can. If the blocks stored by servers encode information about more than one (unrelated) file, then it is more difficult for the enforcers to argue that any particular blocks should be banned. Marc Waldman and David Mazi=E8res have a paper on a design that does this (using a variant of Shamir secret sharing); they call it "entangling" files. It looked to me as though this idea could be added on to almost any system. I'll ask them whether the paper can be put on the web. - -- = David Hopwood Home page & PGP public key: http://www.users.zetnet.co.uk/hopwood/ RSA 2048-bit; fingerprint 71 8E A6 23 0E D3 4C E5 0F 69 8C D4 FA 66 15 0= 1 Nothing in this message is intended to be legally binding. If I revoke a public key but refuse to specify why, it is because the private key has b= een seized under the Regulation of Investigatory Powers Act; see www.fipr.org= /rip -----BEGIN PGP SIGNATURE----- Version: 2.6.3i Charset: noconv iQEVAwUBOozfxTkCAxeYt5gVAQHuZgf/aBpDSvB1pIr9SrwiiX1j841NyVirsFtR V3LOp3PWlRfxkGIT5mxnIvpmTvDeLEpMKcKyrhVB9DJTi4VsyeSOpVyT46B9FmyC C1LkbHaLmvi3S74sHiKeQ5EkmxA9Q0OhMminoSF7gtAurZ7bMa5IEve9XzwBiaQM 3iZxT/v0Bh7kqcmO93VEGLRCyeqCH9rsZhbG/+PIzCihtXPyAfZk24AhAxafxAqY o1S1m3UrBgD7W9tAGcx2iIJbHXhCDtjhyaOwJC4ezXLbbZKmhsd3V971qfP+2FFM SG5GY19AARGSPj4ydzBgRWSYDytvWyFKrOSGTuhE/jnSHg22uPFvDg=3D=3D =3DhzxK -----END PGP SIGNATURE----- From hal@finney.org Sat Feb 17 12:27:41 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 12:27:37 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id JAA17208 for bluesky@franklin.oit.unc.edu; Sat, 17 Feb 2001 09:25:24 -0800 Date: Sat, 17 Feb 2001 09:25:24 -0800 From: hal@finney.org Message-Id: <200102171725.JAA17208@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: the three-services model Wei writes: > I have been assuming a storage system design that decomposes into three > services: > > 1. name lookup service: maping user-friendly names/URLs to content-hash > based ids > 2. data location service: maping content-hash based ids to storage servers > (network addresses and data transport protocols) that can deliver the data > 3. data transport service: actually obtaining content from a storage > server given its address and a content-hash based id Not all services seem to fit this model. Mojonation does not have user friendly names AFAIK. The end result of their publication process is a "dinode" consisting of 8 SHA-1 hashes concatenated. One example is: http://localhost:4004/Dinode/eNpl0stuglAQBuBXIezRUzkCJogBFOQmKFYsGwNyFO VmQUR8-mrF9MLyn8n3ZxbDjq5JjF1QXhyydIi_dQA-4tjxIc0C9DMGncF9gRWJdzoEQ3zh9 I8CszPNCb28gSxMPnwH0sQiTysci1G6P4f3KgpSDM6xdujlCPPjzH_QIkwMay_BbbYKFUOn qBUpvlfmji4NHEvLZIjTePc_qnZSTUs6kTOEbMtH5Lo5n9pz6F32DaLaKCXsYB1pNC-TSlk CBiy3B30r-ycSNKjfRusprzPL8xUyO3GRXdcilGI1VJPN6XUebKN5ZFaWtrbUebS3bqqxEU 4C_5kfqRnfILKNavoqlxpZbatc7glFbToXy9RdxNuv83ptpMWRPwahOOgrK7UoAqcWtdtGZ oLVvEFvbeSDcDp2XEXyILFR6rMr9RN14gtpNGkQeKBldvbib1lwDNv9HZ-FxQyhAAUcZLt_ Mtt9vgv3BbSVu-I/. (I've broken it up into multiple lines.) Likewise I don't think Napster or Gnutella have a URL or name for persistent addressing of data. Instead, these systems are search based. When you want to find data, you do a search providing various keywords of substrings. Data is published on the network with associated meta-data that can include title, author, etc. Your searches use this information to find data that you might want. This is presented to you and you then choose which files to download. Once the search is done you might or might not need to go through steps 2 and 3 above. I think in MojoNation you do go through two more steps. The search gives you the dinode, but that still has to be converted into location information which may involve more lookups (I'm not sure about this part). In Napster and Gnutella I believe the search returns IP addresses which are sufficient locators already. > This results in two levels of indirection. Some current systems seem to > have only one level, either > > a. combining (1) and (2), so that a name is mapped directly to a storage > server, or > b. combining (2) and (3), so that storage servers are addressed by > content-hash base ids. > > Having an extra level of indirection allows problems with both (a) and (b) > to be avoided. The problems with (a) are that the storage server has to be > trusted, and that there is waste if the same data object goes by several > names in different namespaces and is stored on multiple servers. I think the trust issue with (a) is present in step 1 in the three step model. This is a hard problem which has been much discussed on the Freenet list. How do you trust the name to address mapping? Do you need to introduce cryptography, a trust model, a PKI? Such systems have never been successful, although it might be argued that P2P is the "killer ap" for end-user PKI. > The > problems with (b) are that if a data object changes, it would have to be > relocated to a different server, and that servers have little flexibility > in choosing what to store. These problems exist in Freenet, but they don't seem that severe. Data objects in that system are considered fluid and don't have a permanent home, flowing through the network. Any system which will avoid the slashdot effect (what Adam calls flash crowds) needs to be able to migrate and spread data according to demand. As for servers not being able to choose what they store, in Freenet it is a feature not a bug, as it is considered to add to the censorship resistance of the network. Hal From hal@finney.org Sat Feb 17 14:57:40 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 14:57:36 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA17464 for bluesky@franklin.oit.unc.edu; Sat, 17 Feb 2001 11:55:24 -0800 Date: Sat, 17 Feb 2001 11:55:24 -0800 From: hal@finney.org Message-Id: <200102171955.LAA17464@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity David Hopwood writes: > If the blocks stored by servers encode information about more than one > (unrelated) file, then it is more difficult for the enforcers to argue > that any particular blocks should be banned. Marc Waldman and David > Mazières have a paper on a design that does this (using a variant of > Shamir secret sharing); they call it "entangling" files. It looked to > me as though this idea could be added on to almost any system. I'll ask > them whether the paper can be put on the web. I heard the Freehaven developers were looking at this concept. A similar idea was proposed by David Madore [1]: : The principle is very simple. People distribute samples of random : data. We call these samples pads. A pad is a file containing random bits, : completely indistinguishable from white noise, and of a fixed length. I : propose a standard length of 128kB (131072 bytes) for each pad. : ... : Each pad by itself is completely without value. It is a mere hunk of : random data. However, if you combine several pads together by XORing them, : you can recover some data that are hidden in the pads. The information is : in no single pad, but it is somehow delocalized in all the pads together. : : The point is that if a suppressed piece of information can be recovered : from combining n pads (and no less), no single person is distributing : anything of value since all he is doing is givint out a sample of random : data. Indeed, most of them might not even be aware that their pad can : be used to produce the information in question. : : Again, each pad is truly mathematically indistinguishable from : random noise, and it is completely impossible, in the absence of date : information, to know who put the data in a set of pads. In the context of U.S. law, the question is what your obligations are under the DMCA with this system. If you publish a random pad, and you are informed by a copyright owner that your pad, when xor'd with five others, produces some infringing data, are you obligated to take it down? The DMCA does not list "it would destroy other data" as a justification for refusal. One possible point of weakness with this method (and with the hide-the-server approach) is that the censors can go after the name lookup portion. For file entanglement to work there must be an index somewhere that says, for file so-and-so, combine these 5 files from the system. The DMCA lets owners take down links to infringing data, so they could go after this part and then no one could find anything. Hal [1] http://www.eleves.ens.fr:8080/home/madore/misc/freespeech.html From weidai@eskimo.com Sat Feb 17 23:09:24 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 23:09:19 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id UAA13754 for ; Sat, 17 Feb 2001 20:09:12 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id UAA24476 for bluesky@franklin.oit.unc.edu; Sat, 17 Feb 2001 20:09:12 -0800 (PST) Date: Sat, 17 Feb 2001 20:09:12 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: the three-services model Message-ID: <20010217200911.I13542@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Sat, Feb 17, 2001 at 09:25:24AM -0800 On Sat, Feb 17, 2001 at 09:25:24AM -0800, hal@finney.org wrote: > Not all services seem to fit this model. Mojonation does not have user > friendly names AFAIK. The end result of their publication process is a > "dinode" consisting of 8 SHA-1 hashes concatenated. One example is: > http://localhost:4004/Dinode/eNpl0stuglAQBuBXIezRUzkCJogBFOQmKFYsGwNyFO > VmQUR8-mrF9MLyn8n3ZxbDjq5JjF1QXhyydIi_dQA-4tjxIc0C9DMGncF9gRWJdzoEQ3zh9 > I8CszPNCb28gSxMPnwH0sQiTysci1G6P4f3KgpSDM6xdujlCPPjzH_QIkwMay_BbbYKFUOn > qBUpvlfmji4NHEvLZIjTePc_qnZSTUs6kTOEbMtH5Lo5n9pz6F32DaLaKCXsYB1pNC-TSlk > CBiy3B30r-ycSNKjfRusprzPL8xUyO3GRXdcilGI1VJPN6XUebKN5ZFaWtrbUebS3bqqxEU > 4C_5kfqRnfILKNavoqlxpZbatc7glFbToXy9RdxNuv83ptpMWRPwahOOgrK7UoAqcWtdtGZ > oLVvEFvbeSDcDp2XEXyILFR6rMr9RN14gtpNGkQeKBldvbib1lwDNv9HZ-FxQyhAAUcZLt_ > Mtt9vgv3BbSVu-I/. (I've broken it up into multiple lines.) > > Likewise I don't think Napster or Gnutella have a URL or name for > persistent addressing of data. > > Instead, these systems are search based. When you want to find data, you > do a search providing various keywords of substrings. Data is published > on the network with associated meta-data that can include title, author, > etc. Your searches use this information to find data that you might want. > This is presented to you and you then choose which files to download. The three services model doesn't preclude additional services like keyword-based searching, but clearly for a general purpose storage system, persistent and secure naming is a key requirement. Also note that the three services can be designed, implemented, and provided independently of each other, as long as we standardize the interfaces. So although MojoNation may not provide a name service, it can still take advantage of other name services. (Why does it use eight concatenated hashes instead of just one, btw?) > I think the trust issue with (a) is present in step 1 in the three > step model. The idea is to limit this trust issue to only the name service, so that you don't have to trust location or storage servers not to modify the data. > This is a hard problem which has been much discussed on the > Freenet list. How do you trust the name to address mapping? Do you > need to introduce cryptography, a trust model, a PKI? Such systems > have never been successful, although it might be argued that P2P is the > "killer ap" for end-user PKI. I think a distributed trust model is the only thing that can work. But because the three services can be independent, there can actually be multiple name services, using different trust models or PKIs (or even insecure ones that don't use any cryptography). > These problems exist in Freenet, but they don't seem that severe. > Data objects in that system are considered fluid and don't have a > permanent home, flowing through the network. Any system which will avoid > the slashdot effect (what Adam calls flash crowds) needs to be able to > migrate and spread data according to demand. Freenet can't be considered a general purpose storage system either, since it doesn't provide any reliability. There is no way to make reasonably sure that if I publish a document on Freenet, it can be accessed from anywhere, and will be stored for at least a certain amount of time. In this regard it's even worse than the world wide web. > As for servers not being > able to choose what they store, in Freenet it is a feature not a bug, > as it is considered to add to the censorship resistance of the network. Perhaps servers not being able to choose what they store can be considered a feature, but the other side of that coin is that users can't choose which servers to store their data, which is certainly a bug. If servers are addressed by content-hash id, as a user I may be forced to store my data on a server that I know is unreliable or controlled by an attacker. The advantage of seperating the location service from the storage/transport service is to allow users to choose which servers to use for long-term guaranteed storage, while still allowing the data to be accessed from anywhere, and not preventing other servers from caching whatever data they deem most profitable. From weidai@eskimo.com Sat Feb 17 23:25:32 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 17 Feb 2001 23:25:15 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id UAA17036 for ; Sat, 17 Feb 2001 20:25:03 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id UAA25425 for bluesky@franklin.oit.unc.edu; Sat, 17 Feb 2001 20:25:04 -0800 (PST) Date: Sat, 17 Feb 2001 20:25:03 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: User Anonymity Message-ID: <20010217202503.K13542@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Sat, Feb 17, 2001 at 11:55:24AM -0800 On Sat, Feb 17, 2001 at 11:55:24AM -0800, hal@finney.org wrote: > One possible point of weakness with this method (and with the > hide-the-server approach) is that the censors can go after the name > lookup portion. For file entanglement to work there must be an index > somewhere that says, for file so-and-so, combine these 5 files from > the system. The DMCA lets owners take down links to infringing data, > so they could go after this part and then no one could find anything. Let's assume that the name and location services (or just the parts that are subject to censorship), because they doesn't require high bandwidths, can be located in remote jurisdictions. But your other point is still valid. Entanglement is not really a viable defense since the censor can always require you to remove the data regardless of whether it's entangled. Also it's hard to see how a system that uses entanglement will be competitive enough with systems that don't use it to attract non-censored data, which presumably is what you'd want to entangle with. From hal@finney.org Sun Feb 18 02:58:44 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 18 Feb 2001 02:58:40 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id XAA19052 for bluesky@franklin.oit.unc.edu; Sat, 17 Feb 2001 23:56:28 -0800 Date: Sat, 17 Feb 2001 23:56:28 -0800 From: hal@finney.org Message-Id: <200102180756.XAA19052@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: User Anonymity Wei writes: > Entanglement is not really a viable defense since the censor can > always require you to remove the data regardless of whether it's > entangled. Another possible defense is that the material being stored is not the copyrighted data, but scrambled bits. Mojo Nation appeared to be considering a similar approach, according to the Oct 28 article at http://www.thetimes.co.uk/article/0,,26347,00.html: The companyÕs defence lies in the Digital Millennium Copyright Act, which states that Internet service providers cannot be held liable for their usersÕ activities as well as the way the software works. Instead of carrying whole files at a time, Mojo NationÕs software breaks down files into encrypted bite-size pieces which, like flat-pack furniture, are only reassembled once they reach their destination. Transporting files in this way not only makes it easier for users with less powerful computers to download big files, but also means the company is distributing encrypted data rather than content which can be subject to copyright laws. I doubt that this would hold up in court though. The copyright holder would argue that the information is there and can be recovered by end users; the details of how it is stored should not matter. You'd have to look at the specifics of how the laws are worded to be sure. Hal From Ted Anderson Sun Feb 18 17:59:48 2001 Message-ID: <3A9053D8.8C8A172A@transarc.com> Date: Sun, 18 Feb 2001 17:59:36 -0500 From: Ted Anderson Organization: Transarc Corporation X-Mailer: Lyris List Manager Newsgroup Interface MIME-Version: 1.0 Newsgroups: bluesky Subject: Re: the three-services model References: <95285@bluesky> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit To: bluesky I think Wei Dai's separation of components is a good one. We need to be clear exactly what trust each component provides. As Hal said there are various types of trust; it is easy to get confused. Using content hashes as ids provides a trusted name for a bunch of data. It is a shorthand for the data itself. The space and time distribution system doesn't have to be trusted not to mangle the data it is shipping and storing. So the trust that content hashes provide is only that the distribution system is not corrupting the data. There is still the matter of attaching human readable names to content hashes. Both a file system directory and an HTML web page provide mappings between names and URIs and could be linked into a namespace that could be used to attach useful names to content hashes. Ultimately, I don't think there is any avoiding the need to use digital signatures to investing trust in this mapping. You can think of these mappings (e.g. directory entries or web links) as statements. Namely, principal A *says* name N *means* content C. A signature connects "N -> C" to "A", it is essentially a certificate. To decide whether this mapping is useful we need to evaluate our trust in "A". This problem has been much studied in connection with public key systems (such as SDSI[1] and SPKI[2]) and we can use some of their ideas in designing the namespace of the storage system. Separating the naming from the distribution systems this way has an interesting effect on the permission model. Instead of having the file server control read and write access to data, two radically different mechanisms are needed. Read permission is controlled using encryption and carefully handing out the keys. Write permission devolves to control of the namespace. The read case is pretty easy to understand and giving users (forcing them to accept) real control over the data they produce may be a "killer-app" for end-user PKI. The write case is more subtle: what does it mean to write a file? In this model a write changes the "meaning" of a name and hence the mapping between a file's name and some content. This goes directly to the issue of following links and revising history. Sometimes you want the old, original contents that a name mapped to and other times you want the latest mapping of the name to its current contents. In practice, the namespace probably can't keep track of all meanings a name ever had, but it certainly can keep an interesting subset if the owner of the (relevant portion of the) namespace wants to. To support both cases the distribution system should just use content hashes and let users decided, when, whether and how much they trust namespace owners. Ted [1] http://theory.lcs.mit.edu/~cis/sdsi.html [2] http://www.pobox.com/~cme/html/spki.html From hal@finney.org Sun Feb 18 19:53:49 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 18 Feb 2001 19:53:44 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id QAA21642 for bluesky@franklin.oit.unc.edu; Sun, 18 Feb 2001 16:51:33 -0800 Date: Sun, 18 Feb 2001 16:51:33 -0800 From: hal@finney.org Message-Id: <200102190051.QAA21642@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: the three-services model Ted Anderson writes: > There is still the matter of attaching human readable names to content > hashes. Both a file system directory and an HTML web page provide > mappings between names and URIs and could be linked into a namespace > that could be used to attach useful names to content hashes. The same problem arises with search systems. Freenet is using a metadata concept where each document has associated with it a set of attribute-value pairs. These could include title, author, keywords, etc. and be a target for searches. The question is, how do you know that a particular set of metadata accurately describes a document. > Ultimately, I don't think there is any avoiding the need to use digital > signatures to investing trust in this mapping. I would question this on two counts. First, digital signature based systems don't appear to me to be workable, at least in some approaches I've seen. And second, perhaps there are alternatives that will work and scale better. Freenet considered a digital signature approach called mappings. Essentially, individuals would sign the name-to-address (or metadata to CHK) mappings. Each such entry would collect signatures by all the people who had investigated it and found it valid. Then you would list the people whom you had found to give trustworthy signatures on the mappings, and the software would check new mappings to see if they were signed by one of the people you trusted. You could perhaps extend this with some transitive trust model (where people could share their recommendations about whom to trust). I don't think this system can scale. You will have the same problem as with the PGP web of trust, namely that most people you want to communicate with are not signed by anyone you have ever heard of. The number of people in the world and the diversity of their interests makes me doubtful that you can expect to find a signature on a mapping that you will respect. Furthermore the number of signatures that would have to be kept on each mapping is huge. > You can think of these mappings (e.g. directory entries or web links) as > statements. Namely, principal A *says* name N *means* content C. A > signature connects "N -> C" to "A", it is essentially a certificate. To > decide whether this mapping is useful we need to evaluate our trust in > "A". This problem has been much studied in connection with public key > systems (such as SDSI[1] and SPKI[2]) and we can use some of their ideas > in designing the namespace of the storage system. I'm not sure what you are proposing here, whether it is similar to the mappings concept Freenet explored, or perhaps a more direct use of crypto keys to control namespace. Freenet has a key namespace called SVKs (Signature Verification Keys) where a public key hash can be the root of a namespace, and the key holder can create any names he desires within that namespace. You can have a whole heirarchy rooted at a name which is essentially the hash of the public key. (Hence this is not a "friendly" naming system in itself.) This is useful for some purposes but does not solve the general problem of name mapping. The non-crypto idea that Freenet has been exploring is essentially a voting scheme. Whenever you fetch a document you find out whether the name (or metadata) was good. You can then give some feedback about this fact back into the system, which will raise or lower the probability that the mapping will be found in the future. You could have a hybrid scheme with signed mappings, and a voting based PKI similar to that proposed by Stubblebine and Reiter (where the amount you trust a key depends on how many certification paths there are to it). But in any system that will scale, you're going to end up trusting a lot of keys without much justification. In that case it is questionable if the crypto is buying you anything. Hal From hal@finney.org Sun Feb 18 20:00:41 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 18 Feb 2001 20:00:36 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id QAA21672 for bluesky@franklin.oit.unc.edu; Sun, 18 Feb 2001 16:58:25 -0800 Date: Sun, 18 Feb 2001 16:58:25 -0800 From: hal@finney.org Message-Id: <200102190058.QAA21672@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: the three-services model Wei writes: > The three services model doesn't preclude additional services like > keyword-based searching, but clearly for a general purpose storage system, > persistent and secure naming is a key requirement. But perhaps a *friendly* name is not a requirement. Freenet has considered a scheme where files would be described by metadata. The metadata for a file would be a short list of attribute-value pairs for things like title, author, publication date, keywords, etc. You would search using these fields, and then get back a content hash. MojoNation works much the same way. You search on metadata and get back a content hash like the one I showed earlier. These systems suggest that item (1) in the three service model may be modified by having metadata in place of a friendly name. You could also imagine systems which allow direct searches of the text of documents (similar to what we have on the web with search engines) and return a content hash key. These would have neither friendly names nor metadata. Hal From hal@finney.org Mon Feb 19 03:31:19 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 19 Feb 2001 03:31:08 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id AAA22624 for bluesky@franklin.oit.unc.edu; Mon, 19 Feb 2001 00:28:56 -0800 Date: Mon, 19 Feb 2001 00:28:56 -0800 From: hal@finney.org Message-Id: <200102190828.AAA22624@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: the three-services model Wei wrote: > Also note that the > three services can be designed, implemented, and provided independently of > each other, as long as we standardize the interfaces. So although > MojoNation may not provide a name service, it can still take advantage of > other name services. Yes, it would be good to be able to slice up the services "vertically" rather than just "horizontally" as we have them now. It still seems to me that the most natural split is between levels 1 and 2. This is the mapping from names (or perhaps search data) to content hashes. It is a rather different (and perhaps harder) problem than the other layers. I worry that the split between 2 and 3 makes too rigid an assumption about the particular mechanism for finding data - look up the server, and then query the server. I don't think we should preclude the possibility of other ways of finding data. > (Why does it use eight concatenated hashes instead of > just one, btw?) I think it is because MojoNation splits the files into 8 pieces using a Shamir style sharing algorithm, and stores all 8 of the content hashes. Then some number < 8 of them is needed to reconstruct the data. > > This is a hard problem which has been much discussed on the > > Freenet list. How do you trust the name to address mapping? Do you > > need to introduce cryptography, a trust model, a PKI? Such systems > > have never been successful, although it might be argued that P2P is the > > "killer ap" for end-user PKI. > > I think a distributed trust model is the only thing that can work. But > because the three services can be independent, there can actually be > multiple name services, using different trust models or PKIs (or even > insecure ones that don't use any cryptography). This makes a lot of sense. Different projects can explore their own approaches to handling the naming problem. If they could resolve to a common format for the actual data it would let people mix and match. > > These problems exist in Freenet, but they don't seem that severe. > > Data objects in that system are considered fluid and don't have a > > permanent home, flowing through the network. Any system which will avoid > > the slashdot effect (what Adam calls flash crowds) needs to be able to > > migrate and spread data according to demand. > > Freenet can't be considered a general purpose storage system either, since > it doesn't provide any reliability. There is no way to make reasonably > sure that if I publish a document on Freenet, it can be accessed from > anywhere, and will be stored for at least a certain amount of time. In > this regard it's even worse than the world wide web. No system is perfectly reliable. Freenet will provide some level of reliability. Whether it is able to evolve into a sufficiently reliable system for general use remains to be seen. The OceanStore system at http://oceanstore.cs.berkeley.edu is a more complex and sophisticated system which has concepts similar to Freenet. Here is an excerpt of their ideas on "nomadic" data: In a system as large as OceanStore, locality is of extreme importance. Thus, we have as a goal that data can be cached anywhere, anytime. We call this policy promiscuous caching. Data which is allowed to flow freely is called nomadic data. Note that nomadic data is an extreme consequence of separating information from its physical location. Although promiscuous caching complicates data coherence and location, it provides great flexibility to optimize locality and to trade off consistency for availability. OceanStore keeps careful track of where data is located as it flows through the network. This allows its data lookup algorithm to be highly reliable and fault tolerant. My concern is that it may turn out to be costly to keep location information up to date, and since they don't have any code on their web page, it's hard to judge how the efficiency tradeoffs will turn out. > Perhaps servers not being able to choose what they store can be considered > a feature, but the other side of that coin is that users can't choose > which servers to store their data, which is certainly a bug. If servers > are addressed by content-hash id, as a user I may be forced to store my > data on a server that I know is unreliable or controlled by an attacker. > > The advantage of seperating the location service from the > storage/transport service is to allow users to choose which servers to use > for long-term guaranteed storage, while still allowing the data to be > accessed from anywhere, and not preventing other servers from caching > whatever data they deem most profitable. I'm not sure I follow how this separation achieves these goals. How would caching work in this system? Is it a feature of level 2 or level 3? I thought in your model I would I do a lookup in level 2 with the CHK to get a host, and from what you say here I gather that this would be the host chosen by the author to hold the data inserted. Then I go to level 3 and contact that host and request the data. There don't seem to be opportunities for data caching in level 2 since no data flows there, and not in level 3 since you talk directly to the machine where the data was originally stored. Hal From ross.anderson@cl.cam.ac.uk Mon Feb 19 08:47:07 2001 X-Mailer: Lyris Web Interface Date: Mon, 19 Feb 2001 08:47:03 -0500 Subject: Eternity etc To: bluesky@ibiblio.org From: "Ross" I've reorganised my home page to bring together the Eternity paper and various other things I've done that are relevant to peer-to-peer. See http://www.cl.cam.ac.uk/~rja14/#Peer-to-Peer There will also be more in my book when it comes out in a couple of weeks' time: http://www.cl.cam.ac.uk/~rja14/book.html Enjoy! Ross From weidai@eskimo.com Mon Feb 19 10:09:08 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 19 Feb 2001 10:09:01 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id HAA20140 for ; Mon, 19 Feb 2001 07:08:47 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id HAA16734 for bluesky@franklin.oit.unc.edu; Mon, 19 Feb 2001 07:08:47 -0800 (PST) Date: Mon, 19 Feb 2001 07:08:47 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: the three-services model Message-ID: <20010219070847.B4445@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Mon, Feb 19, 2001 at 12:28:56AM -0800 On Mon, Feb 19, 2001 at 12:28:56AM -0800, hal@finney.org wrote: > I think it is because MojoNation splits the files into 8 pieces using a > Shamir style sharing algorithm, and stores all 8 of the content hashes. > Then some number < 8 of them is needed to reconstruct the data. It seems like a more efficient approach would be to identify each share by a hash of the hashes of the shares, and a share number. This way you only need one hash to locate all of the shares. > OceanStore keeps careful track of where data is located as it flows > through the network. This allows its data lookup algorithm to be highly > reliable and fault tolerant. My concern is that it may turn out to be > costly to keep location information up to date, and since they don't > have any code on their web page, it's hard to judge how the efficiency > tradeoffs will turn out. OceanStore actually fits the three-services model pretty well. For names it uses SDSI-style locally linked name spaces. For data location it uses a combination of a quick scheme for popular data, and a slower but more reliable scheme for all data. It's as if Freenet nodes in addition to doing everything else also report what they are storing to a distributed location database. I think the particular location database design used by OceanStore is vulnerable to active attacks (in order to censor a document by making it unfindable), but it doesn't seem very costly. I'll try to elaborate on the attack later. > I'm not sure I follow how this separation achieves these goals. How would > caching work in this system? Is it a feature of level 2 or level 3? > I thought in your model I would I do a lookup in level 2 with the CHK > to get a host, and from what you say here I gather that this would be > the host chosen by the author to hold the data inserted. Then I go to > level 3 and contact that host and request the data. There don't seem to > be opportunities for data caching in level 2 since no data flows there, > and not in level 3 since you talk directly to the machine where the data > was originally stored. What I envision is that storage servers will get usage data from location servers to see which documents are popular, and retrieve and store them deliberately. The user agent can also cache copies of data the user requests and participate in the storage/transport service. (BTW, I prefer to use the names of the services instead of the numbers in order to avoid confusion later when the numbers might be used for something else.) From adam@cypherpunks.ai Wed Feb 21 21:00:20 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 21 Feb 2001 21:00:17 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 552456E; Wed, 21 Feb 2001 22:00:10 -0400 (AST) Date: Wed, 21 Feb 2001 22:00:09 -0400 From: Adam Back To: bluesky@franklin.oit.unc.edu Subject: idealised properties? Message-ID: <20010221220009.A96175@cypherpunks.ai> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i Return-Path: Sender: adam@cypherpunks.ai The discussions we've had so far about threat models highlight the difficulty of finding a threat model which results in a system which is plausible. Anyone want to have a stab at describing idealised model of what one is trying to achieve with censor resistant distributed file systems, without tying anything back to implementation ideas. Adam From graydon@venge.net Wed Feb 21 23:55:47 2001 Return-Path: Received: from spitfire.velocet.net ([209.167.225.66]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 21 Feb 2001 23:55:44 -0500 Received: from venge.net (H42.C192.tor.velocet.net [216.138.192.42]) by spitfire.velocet.net (Postfix) with SMTP id 7DE1A19A077 for ; Wed, 21 Feb 2001 23:55:32 -0500 (EST) Received: (qmail 29968 invoked by uid 1000); 22 Feb 2001 04:49:09 -0000 From: graydon@venge.net Date: Wed, 21 Feb 2001 23:49:09 -0500 To: Global-Scale Distributed Storage Systems Subject: Re: idealised properties? Message-ID: <20010221234908.A29954@venge.net> Reply-To: graydon@pobox.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i In-Reply-To: ; from adam@cypherspace.org on Wed, Feb 21, 2001 at 10:00:09PM -0400 the discussions I've seen focus on 3 categories of threat model: * incompetence, misconfiguration, congestion, accidental deletion, disconnection * un-coordinated attack by disagreeable people: script kiddies, lawyers with small-ish clients, ex-roommates * systemic attack by global organizations with arbitrary resources. the third is appealing to many crypto people (me too) because it's so cloak-and-dagger, but it's hard to deal with because they can resort to law-making, treaties, rubber hoses, etc. the first and second are imho hard enough to deal with and can probably offer enough utility to be worthwhile. -graydon From adam@cypherpunks.ai Thu Feb 22 13:51:43 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 22 Feb 2001 13:51:38 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id AD5EB84; Thu, 22 Feb 2001 14:51:31 -0400 (AST) Date: Thu, 22 Feb 2001 14:51:31 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: idealised properties? Message-ID: <20010222145131.B5002@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from graydon@venge.net on Wed, Feb 21, 2001 at 11:49:09PM -0500 Return-Path: Sender: adam@cypherpunks.ai So I was presuming the 3rd model too, because it's by far the most interesting a system to have for positive social implications. So if we consider that threat model I was thinking we could described idealised requirements without focussing on how one could implement those requirements. So for example: - homegenous system with graceful degradation -- ie you should be able to continue using the system and all information should remain as individual nodes are removed. Homogenous so no node are fatter targets than any other nodes. This maximises cost of attacking the system. - servers should not be able to see what data they are serving or at least not know without some digging to figure it out - possibly users should not be able to find out which server they got information from, though this seems more like an design issue. You could for example imagine designs where no attempts to prevent this are made, and yet the other two criteria were met. (eg. my hot potato suggestion, though it's not clear how to build that in a way that performs efficiently). Adam On Wed, Feb 21, 2001 at 11:49:09PM -0500, graydon@venge.net wrote: > the discussions I've seen focus on 3 categories of threat model: > > * incompetence, misconfiguration, congestion, accidental deletion, > disconnection > > * un-coordinated attack by disagreeable people: script kiddies, > lawyers with small-ish clients, ex-roommates > > * systemic attack by global organizations with arbitrary resources. > > the third is appealing to many crypto people (me too) because it's so > cloak-and-dagger, but it's hard to deal with because they can resort to > law-making, treaties, rubber hoses, etc. the first and second are imho > hard enough to deal with and can probably offer enough utility to be > worthwhile. From hal@finney.org Thu Feb 22 14:41:00 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 22 Feb 2001 14:40:57 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA08387 for bluesky@franklin.oit.unc.edu; Thu, 22 Feb 2001 11:38:46 -0800 Date: Thu, 22 Feb 2001 11:38:46 -0800 From: hal@finney.org Message-Id: <200102221938.LAA08387@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: idealised properties? Graydon wrote: > the discussions I've seen focus on 3 categories of threat model: > > * incompetence, misconfiguration, congestion, accidental deletion, > disconnection > > * un-coordinated attack by disagreeable people: script kiddies, > lawyers with small-ish clients, ex-roommates > > * systemic attack by global organizations with arbitrary resources. I agree that the third attack model, as written, is so broad as to make any resistance impossible. At the extreme the organizations can simply kill every man, woman and child on earth and their problem is solved. Clearly this formulation is not a useful way to proceed. I would suggest an intermediate model, although it is hard to define precisely. Basically it is a constrained version of number three, where the attack is limited by political considerations. In other words, "arbitrary" resources are not available. There are a variety of constraints that we have discussed. - Attackers who are unable to interfere with systems in certain political jurisdictions, and unable to prevent communication with those jurisdictions. - Attackers who are constrained by legislative protections on freedom of speech and who are unable to interfere with encrypted communications. - Copyright-protecting attackers who are constrained by the U.S. DMCA which provides certain safe harbors for communication. - Cost-based attackers who are unwilling to legally pursue individual users but who will go after vulnerable points where large numbers of users can be shut down at once. I didn't come up with these threat models purely by considering what attackers might do. Rather, I took various features of existing and proposed systems and looked at the kinds of attacks these systems are vulnerable to, or are designed to resist. In other words, I inferred the threat model from the characteristics of the system. This may seem like a backwards way to go about it, but given the complexity of the real world situation, I think it is a useful guide to what is possible and what is not. Hal From hal@finney.org Fri Feb 23 18:05:53 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 23 Feb 2001 18:05:49 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id PAA13895 for bluesky@franklin.oit.unc.edu; Fri, 23 Feb 2001 15:03:38 -0800 Date: Fri, 23 Feb 2001 15:03:38 -0800 From: hal@finney.org Message-Id: <200102232303.PAA13895@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Encrypting stored data There seem to be two purposes for encrypting data stored in a globally shared data space. One is to allow for "private" data which is only accessible to one person or a limited group. The extreme case is when the shared data space is used for personal file backup and only the file owner can decrypt it. A more widely shared file might be one which is used by a work group. The other purpose is as an anti censorship measure. In this case the decryption keys are generally expected to be widely distributed along with the pointer to the file. The encryption is used specifically to prevent server owners from being able to know what files they are storing. Publius [1] does this, as does Freenet [2]. I don't think MojoNation [3] encrypts stored data, but it does split it up using a cryptographic technique which provides much the same effect, that a server can't tell what data it is storing. I think the original Eternity service concept by Ross Anderson [4] may have included this kind of server blindness as well. This second purpose is controversial, because it relates to a specific threat model which may not be realistic in the long term. It prevents servers from being coerced into removing data based on structural characteristics of the data. For example, servers might be required to remove text files which had certain strings in them, or a server operator might be blamed if an image file showed forbidden material. What these systems can't prevent is orders to remove data which matches a specific bit pattern. Even if the data is encrypted or split into pieces, it will often still be possible for those seeking to suppress the material to learn the encrypted or split form of the data and order it removed. A related issue is whether server operators ought to be able to keep certain material off their systems if they want. Freenet and Publius intentionally prevent server operators from doing this. Other systems may take the opposite position and want to allow operators to know what data they are serving. Hal [1] http://www.cs.nyu.edu/~waldman/publius/ [2] http://freenet.sourceforge.net/ [3] http://www.mojonation.net/ [4] http://www.cl.cam.ac.uk/users/rja14/eternity/eternity.html From hal@finney.org Sun Feb 25 20:56:48 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 25 Feb 2001 20:56:39 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id RAA04264 for bluesky@franklin.oit.unc.edu; Sun, 25 Feb 2001 17:53:50 -0800 Date: Sun, 25 Feb 2001 17:53:50 -0800 From: hal@finney.org Message-Id: <200102260153.RAA04264@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Links Things are moving fast in the P2P world. As most of us will have heard, the recording industry is now going after OpenNap servers, armed with the favorable appellate court decision in the Napster case. Many observers expect U.S. based OpenNap servers to fold. This will be a good preview of how well "jursidiction shopping" works on a widely used P2P system. Here are a few links which I have found to try to keep up with what is happening: http://www.peertal.com/ This is a slashdot style site which depends upon stories and links submitted by users. There is not much discussion and many of the stories are business oriented. It gets about 2 to 3 stories a day. http://www.openp2p.com/ This is the O'Reilly P2P portal. They were the sponsors of the P2P conference held a couple of weeks ago. It has a very large directory of P2P efforts, both commercial and free, usefully categorized as file-sharing, distributed computation, super-distribution, etc. There is also a news portal but it only gets 1-2 stories a week. http://www.infoanarchy.org/ This is another slashdot style news portal with a bit of an underground flavor. Activity level varies from a couple of stories a week to several a day. There are a number of other sites but these are ones I have found most useful. Other suggestions would be appreciated. Hal From hal@finney.org Sun Feb 25 23:13:49 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 25 Feb 2001 23:13:43 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id UAA05946 for bluesky@franklin.oit.unc.edu; Sun, 25 Feb 2001 20:10:55 -0800 Date: Sun, 25 Feb 2001 20:10:55 -0800 From: hal@finney.org Message-Id: <200102260410.UAA05946@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Scalability One of the controversial technical issues in P2P development is scalability. Recently a paper by one of the Napster technical leads [1] analyzed the behavior of a Gnutella network mathematically and found that it would not scale up to the size of Napster (target of 1 million users). There was some discussion on Slashdot [2] and a rebuttal [3] offered by a developer of the Gnutella client LimeWire. Basically he argued that Gnutella searches would only try to reach a few thousand nodes regardless of how many are on the network, which would reduce its usefulness IMO. Are there inherent scalability issues with P2P networks? Freenet goes to some lengths to try to locate data while using only a small amount of network traffic. This is intended to avoid the need for a centralized database of where all data is stored. Another strategy is be to use a decentralized database which indexes the data, which I think is how MojoNation works, using "content trackers" which any peer node can run. However this raises the issue of how you locate the index nodes. MojoNation has a MetaTracker which I think is there to help you find content trackers. The MetaTracker is currently centralized but I believe they plan to make it decentralized, but then how will you find them? And what will qualify a node to operate a MetaTracker? I'd like to see discussion of technical solutions to the problems of finding data without swamping the network. It would be especially good if this could handle a search search, something that Freenet does not even try to tackle yet. Mojo handles searching using the same content trackers that store file locations. Hal [1] http://www.monkey.org/~dugsong/mirror/gnutella.html [2] http://slashdot.org/articles/01/02/14/190225.shtml [3] http://www.gnutellanews.com/article.html?id=4206 From ota@transarc.com Mon Feb 26 07:36:19 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 07:36:15 -0500 Received: from po2.transarc.com.transarc.com (po2.transarc.ibm.com [9.38.192.6]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id HAA14986; Mon, 26 Feb 2001 07:30:44 -0500 (EST) Received: by po2.transarc.com.transarc.com (8.9.0.Beta3/3.15) id ; Mon, 26 Feb 2001 07:36:09 -0500 (EST) Received: via switchmail; Mon, 26 Feb 2001 07:36:08 -0500 (EST) Received: from smithfield via qmail ID ; Mon, 26 Feb 2001 07:35:57 -0500 (EST) Date: Mon, 26 Feb 2001 07:35:55 -0500 (EST) Message-Id: To: hal@finney.org Subject: Re: the three-services model CC: bluesky@franklin.oit.unc.edu In-Reply-To: References: From: Ted Anderson On 18 Feb 2001 16:51:33 PST regarding "the three-services model" hal@finney.org wrote: > Ted Anderson writes: > > There is still the matter of attaching human readable names to > > content hashes. ... > The same problem arises with search systems. Freenet is using a > metadata concept where each document has associated with it a set of > attribute-value pairs. These could include title, author, keywords, > etc. and be a target for searches. The question is, how do you know > that a particular set of metadata accurately describes a document. I think it is certainly reasonable to attach metadata to an object either directly or by CHK reference, but this, of course, just puts off the problem. Do you trust a particular collection of metadata annotations or not? > > Ultimately, I don't think there is any avoiding the need to use > > digital signatures to investing trust in this mapping. > I would question this on two counts. First, digital signature based > systems don't appear to me to be workable, at least in some approaches > I've seen. And second, perhaps there are alternatives that will work > and scale better. This is a key (:-) area of disagreement that we should endeavor to resolve. I believe that an important benefit of a decentralized storage system is that it puts security in the hands of the data producers. They control the data by encrypting it and managing the keys as they see fit. They do not *have to* rely on the security of a centralized file server to protect their data; though they *can* entrust the encryption keys to a key server if centralized management is desirable. The trade-off is that revocation of permission is difficult: data must be re-encrypted in a new key. However, this difficulty only mirrors the state of the real world. If you entrust your data to someone, you have to assume they will keep a copy for themselves if they want to. This is just caching and replication after all. Any scalable scheme for end-to-end encryption by individual data producers and consumers is going to depend on public keys, which then can be easily used for digital signatures. > Freenet considered a digital signature approach called mappings. > Essentially, individuals would sign the name-to-address (or metadata > to CHK) mappings. Each such entry would collect signatures by all the > people who had investigated it and found it valid. Then you would > list the people whom you had found to give trustworthy signatures on > the mappings, and the software would check new mappings to see if they > were signed by one of the people you trusted. You could perhaps > extend this with some transitive trust model (where people could share > their recommendations about whom to trust). > I don't think this system can scale. You will have the same problem > as with the PGP web of trust, namely that most people you want to > communicate with are not signed by anyone you have ever heard of. The > number of people in the world and the diversity of their interests > makes me doubtful that you can expect to find a signature on a mapping > that you will respect. Furthermore the number of signatures that > would have to be kept on each mapping is huge. I agree that this system cannot scale without some structure imposed upon it. Instead of having anyone and everyone sign everything they believe, a network of specialists would evolve to add value by creating meaningful names for objects or evaluating useful metadata describing them. I imagine that several services would provide top-level directory trees which provide convenient and logical organizations for most popular categories of data. The leaves of these top-level trees would delegate well-known names to subsidiary organizations (e.g. /com/us/microsoft would map to a value updated periodically to reflect Microsoft's desired content). Competing outfits might offer alternative layouts or specialize in particular subjects (e.g. /entertainment/music/jazz/top-ten). Individual users would adopt one or more of these under their private root so that they can obtain access to global resources. In addition they can organize private parts of their namespace as they see fit, such as recently referenced objects, or favorite reviewers and commentators. In the www domain, this would be like a combination of history list, bookmarks, and a weblog. To handle dynamic content some objects, especially the leaf nodes in the top-level trees, might really be certificates, containing a public key, a CHK for a recent object instance, and a list of server addresses that would provide signed, up-to-date contents. > > You can think of these mappings (e.g. directory entries or web > > links) as statements. Namely, principal A *says* name N *means* > > content C. A signature connects "N -> C" to "A", it is essentially > > a certificate. ... > I'm not sure what you are proposing here, whether it is similar to the > mappings concept Freenet explored, or perhaps a more direct use of > crypto keys to control namespace. Freenet has a key namespace called > SVKs (Signature Verification Keys) where a public key hash can be the > root of a namespace, and the key holder can create any names he > desires within that namespace. You can have a whole heirarchy rooted > at a name which is essentially the hash of the public key. (Hence > this is not a "friendly" naming system in itself.) This is useful for > some purposes but does not solve the general problem of name mapping. My proposal is somewhat like this, except I would use the certificates, described above, instead of special casing hashes of public keys. In a similar way, related groups of name mappings are collected into "directories" and signed by specialists at multiple levels. Ultimately, the user decides who he trusts to provide name mappings in his personal root. I assume that reputation and widely used conventions will provide order to the global namespace, so that most people will have the same names for most objects. > The non-crypto idea that Freenet has been exploring is essentially a > voting scheme. Whenever you fetch a document you find out whether the > name (or metadata) was good. You can then give some feedback about > this fact back into the system, which will raise or lower the > probability that the mapping will be found in the future. This could work as long as someone with a well-known key and a reputation collects the votes and periodically publishes the results. Without this it seems that the scheme has no defense against manipulation. > You could have a hybrid scheme with signed mappings, and a voting > based PKI similar to that proposed by Stubblebine and Reiter (where > the amount you trust a key depends on how many certification paths > there are to it). But in any system that will scale, you're going to > end up trusting a lot of keys without much justification. In that > case it is questionable if the crypto is buying you anything. This is a pretty fundamental issue that needs exploration. I don't claim to be an expert on the issue by any means, however, I don't think the situation is this hopeless. In the SPKI (or is it SDSI?) model, every name mapping is relative to the key which signs it: the "principal A *says* name N *means* content C" business. This limitation bounds the trust problem. You need a few widely trusted keys, redundantly signing a few top-level directories and the rest takes care of itself in a cross-linked flexible hierarchy of names and keys certifying other names and keys. What crypto buys you is a way to understand the trust you invest in these links. The reputation of the key depends on the care with which it is used to sign things. While this isn't perfect, it is a lot more than you have without crypto. Ted Anderson From weidai@eskimo.com Mon Feb 26 08:49:39 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 08:49:31 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id FAA18902 for ; Mon, 26 Feb 2001 05:49:20 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id FAA05589 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 05:49:19 -0800 (PST) Date: Mon, 26 Feb 2001 05:49:19 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: market-based data location (was Scalability) Message-ID: <20010226054915.A446@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Sun, Feb 25, 2001 at 08:10:55PM -0800 On Sun, Feb 25, 2001 at 08:10:55PM -0800, hal@finney.org wrote: > I'd like to see discussion of technical solutions to the problems of > finding data without swamping the network. It would be especially good > if this could handle a search search, something that Freenet does not > even try to tackle yet. Mojo handles searching using the same content > trackers that store file locations. Have you looked at OceanStore's distributed data location algorithm? It's described as a variation on Plaxton et. al.'s randomized hierarchical distributed data structure. I think it should handle scalability quite well, but is weak against denial of service attacks (censorship). Designing a data location algorithm that is both scalable and censorship-resistant seems to be a fairly difficult problem. In OceanStore's scheme, each data object is assigned a number of root nodes responsible for tracking its locations. This assignment is based on the closeness of salted hashes of the object and the IDs of the nodes, which are also hashes. The attack is just to generate potential node IDs until you obtain IDs that would be closest to the salted hashes of the target object, and then start servers with those IDs. Those servers become roots for the object, and you can just instruct them to respond to all requests for it with a reply of "no known location." I think the final solution for the data location problem will have to be market-based, involving both payment and reputation schemes. Without a payment system you can't prevent an attacker from simply overwhelming a server with requests, and there is no incentive for people to contribute resources into the system instead of just leaching from it. Without a reputation scheme, attackers can introduce their own servers and supply the system with bad data, either to censor some data or to cheat people out of their money, and users will have no way of avoiding these corrupt servers. The idea I'm playing with (and have been discussing with Ted Anderson before creating this list) is for each data location server to self select, based on market conditions, some criteria for what data to track, for example any data located in a radius of x from itself, and has a (content-hash based) GUID inside a radius of y from it's own GUID. It would then accept paid or unpaid advertisement from data storage servers, and answer paid or unpaid queries for data location. It can also query other data location servers to fill its own database. Besides tracking data, it would also track other location servers whose ranges overlap with its own, so that if it can't answer a query directly it will return a list of other location servers (along with their tracking and pricing policies) who might be able to or at least get you closer to such a server, and attest to their reliability and comprehensiveness. With this system you would see locations servers that are specialized for different market niches. Some would be neighborhood trackers, with a small network radius and a large GUID radius. Some would be global tracks, with a large network radius and a small GUID radius. Some would be highly priced and specialize in censored data. Some would acquire a reputation for reliability and make money from long-term paid advertisements. Some would have a low query price and make money from queries for popular data. Some would only track other location servers and essentially provide a reputation service. It would be really cool to watch this market self-organize. I think that almost certainly only some scheme that looks like this can scale and be censorship-resistant. The hard part is to work out what strategies the location servers, storage servers, and user agents should use to avoid being cheated and to maximize profits or at least break even, and to build the necessary support infrastructure, for example interface standards and payment systems. From weidai@eskimo.com Mon Feb 26 08:56:45 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 08:56:42 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id FAA20701 for ; Mon, 26 Feb 2001 05:56:36 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id FAA05883 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 05:56:36 -0800 (PST) Date: Mon, 26 Feb 2001 05:56:36 -0800 From: Wei Dai To: bluesky@franklin.oit.unc.edu Subject: list membership Message-ID: <20010226055636.B446@eskimo.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i This list currently has about 24 subscribers. I think it would probably be a good thing to recruit some more people to join our discussions. I believe Ted and I only sent out that one initial invitation, but this is an open list, so please feel free to forward the invitation to other people or other mailing lists, if you think they would be interested. From ota@transarc.com Mon Feb 26 12:02:05 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 12:02:00 -0500 Received: from po2.transarc.com.transarc.com (po2.transarc.ibm.com [9.38.192.6]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id LAA13724; Mon, 26 Feb 2001 11:56:29 -0500 (EST) Received: by po2.transarc.com.transarc.com (8.9.0.Beta3/3.15) id ; Mon, 26 Feb 2001 12:01:53 -0500 (EST) Received: via switchmail; Mon, 26 Feb 2001 12:01:53 -0500 (EST) Received: from smithfield via qmail ID ; Mon, 26 Feb 2001 12:00:38 -0500 (EST) Date: Mon, 26 Feb 2001 12:00:36 -0500 (EST) Message-Id: <8uacioI99g1Q02LU40@transarc.com> To: bluesky@franklin.oit.unc.edu, hal@finney.org Subject: Re: Links / BlueSky webpage update In-Reply-To: References: From: Ted Anderson I went through the mailing list archive this weekend and pulled out a few pointers to other resources that people have posted and updated the BlueSky webpage[1] with a references section. This is still very rough, so I'd be happy if people would send me comments, updates and additions. One thing I haven't done is add a section on the political issues on which Hal has been providing information. Rather than try to keep the BlueSky page current in this rapidly evolving area, I'd like to include a pointer to some other page that someone is maintaining anyway. Hal perhaps you have a page like this? Or you'd like to nominate one or a few good places to direct parties interested in more extensive and current details? Ted [1] http://www.transarc.com/~ota/bluesky/index.html From hal@finney.org Mon Feb 26 13:30:55 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 13:30:50 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id KAA08261 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 10:28:03 -0800 Date: Mon, 26 Feb 2001 10:28:03 -0800 From: hal@finney.org Message-Id: <200102261828.KAA08261@finney.org> To: bluesky@franklin.oit.unc.edu Subject: DMCA on jurisdiction shopping I was looking again at the text of the DMCA on the EFF web site [1] and I noticed that it had something to say about "jurisdiction shopping," the idea that controversial material could be made available worldwide from jurisdictions where it is not forbidden. Section 512(a) immunizes service providers from liability who are just passing the data through in an automated way. It is clearly meant to apply to the owners of the infrastructure through which Internet data passes, and possibly to the ISPs at one or both ends as well. However there is a caveat, which is that injunctive relief as provided in subsection (j) is still applicable. I.e., the court can still order the following: `(B) If the service provider qualifies for the limitation on remedies described in subsection (a), the court may only grant injunctive relief in one or both of the following forms: `(i) An order restraining the service provider from providing access to a subscriber or account holder of the service provider's system or network who is using the provider's service to engage in infringing activity and is identified in the order, by terminating the accounts of the subscriber or account holder that are specified in the order. `(ii) An order restraining the service provider from providing access, by taking reasonable steps specified in the order to block access, to a specific, identified, online location outside the United States. This latter provision applies specifically to the case where infringing data is being made available from outside the United States. The DMCA authorizes courts to issue injunctions forbidding Internet service providers from allowing access to specified online locations outside the U.S. These would presumably be those sites which are providing infringing material. This will severely impair the usefulness of jurisdiction shopping for users in the United States. Any offshore sites which become known as locations where infringing material is available can be blocked. The DMCA goes on to offer the following guidelines for the issuance of such injunctions: `(2) CONSIDERATIONS- The court, in considering the relevant criteria for injunctive relief under applicable law, shall consider-- `(A) whether such an injunction, either alone or in combination with other such injunctions issued against the same service provider under this subsection, would significantly burden either the provider or the operation of the provider's system or network; `(B) the magnitude of the harm likely to be suffered by the copyright owner in the digital network environment if steps are not taken to prevent or restrain the infringement; `(C) whether implementation of such an injunction would be technically feasible and effective, and would not interfere with access to noninfringing material at other online locations; and `(D) whether other less burdensome and comparably effective means of preventing or restraining access to the infringing material are available. It might be possible to design a system which makes it difficult to block access to a foreign site under these guidelines. For example, if data were very widely replicated overseas, and provided at servers which also had non-infringing data, the court would find it harder to justify the injunction. Ultimately though it will come down to the judgement of the court, and in many cases the judges will favor established corporate interests over those of "hacker pirates." Hal [1] http://www.eff.org/ip/DMCA/hr2281_dmca_law_19981020_pl105-304.html From ian@technic.uprizer.com Mon Feb 26 17:56:31 2001 Return-Path: Received: from technic.uprizer.com ([4.18.49.34]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 26 Feb 2001 17:56:27 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f1QN3Bx03294 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 15:03:11 -0800 Date: Mon, 26 Feb 2001 15:03:10 -0800 From: Ian Clarke To: bluesky@franklin.oit.unc.edu Subject: Hi, and some comments Message-ID: <20010226150310.A3230@technic.uprizer.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="7JfCtLOvnd9MIVvH" Content-Disposition: inline User-Agent: Mutt/1.2.5i --7JfCtLOvnd9MIVvH Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Hi All, I joined this list on Hal's suggestion, and looking through the archives there seems to be some interesting discussion. To those of you who don't know, I am the author of the paper "A Distributed Decentralized Information Storage and Retrieval System" upon which Freenet is based, and am the coordinator of the ongoing Freenet development effort (which has extended my original ideas far beyond what is described in the paper). Anyone interested in learning more about Freenet should take a look at some publication's to be found at: http://freenetproject.org/index.php?page=3Dsect_Publications I note with interest the conversation around Freenet's various naming schemes, particularly "Content Hash Keys". Correct me if I have missed something, but the list may also be interested in some of our other key-types. "Keyword Signed Keys" are simple free-form keys which the user can choose, but are rather insecure (an untrustworthy node could intercept and manipulate a response to a request for a KSK). More interestingly, Sub-Space Keys make use of public-private key cryptography to allow people to define a private "subspace" within the Freenet key-space, to which only they can insert keys, thus allowing people to assign a level of trust to all documents within a subspace. Rather than repeat stuff here - anyone interested can find out more about these different key-types at: http://freenetproject.org/index.php?page=3Dicsi-revised#SECTION000310000000= 00000000 There has also been commentary on the fact that Freenet drops unpopular files. This is a frequent topic of discussion on Freenet's mailing lists too, but we long-ago agreed that the purpose of Freenet was never to be a robust file repository, any more than broadcasting something over the radio is a reliable means to ensure it will be recorded for posterity. =20 I notice that people frequently neglect to bear-in-mind the goals of a system when criticising features of it. If Freenet didn't drop files then it could eventually become full, which would be unacceptable, and would provide a trivial attack against it. A MojoNation-style economy is not acceptable either as we believe that people should be able to publish information regardless of their personal wealth. There are other reasons too. Regards, Ian. --7JfCtLOvnd9MIVvH Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6muCuQtgxRWSmsqwRAmGPAJ0QejgH5SRPlFLRhqKHoziiZui8awCfUacL apJO3g6yESzReWXXq0oCdeI= =hrJa -----END PGP SIGNATURE----- --7JfCtLOvnd9MIVvH-- From adam@cypherpunks.ai Tue Feb 27 00:00:45 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 00:00:33 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 7C73249; Tue, 27 Feb 2001 01:00:27 -0400 (AST) Date: Tue, 27 Feb 2001 01:00:27 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: Scalability Message-ID: <20010227010027.C62310@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from hal@finney.org on Sun, Feb 25, 2001 at 08:10:55PM -0800 Return-Path: Sender: adam@cypherpunks.ai I'd guess there would be two systems you'd use: - a system to find the URLs you want - a system to locate a copy of and download the URL content Steve Schear and I were discussing the first one and another approach is to just let a search engine do it for you. Either by exposing the URLs to the real web through gateways which make it appear (to search engines only perhaps) as static content. Or by encouraging people to set up search engines on the disitributed URL space. People find things through other peoples links to them, which is ultimately the same way that the spiders do. Locating a copy of a target URL to download is a different problem. Graydon's log map seems like a reasaonable approach. I guess it depends partly how much data one expects to be stored. What fraction of available space does the content represent. (I'm thinking it could be a very small fraction of available (meaning available and currently online) space as "disk is cheap"). The a low fraction helps as you can "overcache" or mirror. Say you had a mirroring algorithm with random cache replacement policy, and some distributed way for new data to distribute through the system. (You might want to weight caches by popularity, which is what my earlier discussion of hashcash was about). A small fraction makes graceful degradation possible also, as the data is inherently there many times over in a randomly distributed fashion. If you used massive overcacheing with Graydon's log map and algorithm which only admits to holding data after it has ensured secondary copies, the censors could chase references around indefinately. Adam On Sun, Feb 25, 2001 at 08:10:55PM -0800, hal@finney.org wrote: > Gnutella searches would only try to reach a few thousand nodes regardless > of how many are on the network, which would reduce its usefulness IMO. > > Are there inherent scalability issues with P2P networks? Freenet goes > to some lengths to try to locate data while using only a small amount of > network traffic. This is intended to avoid the need for a centralized > database of where all data is stored. > > Another strategy is be to use a decentralized database which indexes the > data, which I think is how MojoNation works, using "content trackers" > which any peer node can run. However this raises the issue of how you > locate the index nodes. MojoNation has a MetaTracker which I think is > there to help you find content trackers. The MetaTracker is currently > centralized but I believe they plan to make it decentralized, but > then how will you find them? And what will qualify a node to operate > a MetaTracker? > > I'd like to see discussion of technical solutions to the problems of > finding data without swamping the network. It would be especially good > if this could handle a search search, something that Freenet does not > even try to tackle yet. Mojo handles searching using the same content > trackers that store file locations. > > Hal > > [1] http://www.monkey.org/~dugsong/mirror/gnutella.html > [2] http://slashdot.org/articles/01/02/14/190225.shtml > [3] http://www.gnutellanews.com/article.html?id=4206 > > --- > You are currently subscribed to bluesky as: adam@cypherspace.org > For list information visit http://www.transarc.ibm.com/~ota/bluesky/ From hal@finney.org Tue Feb 27 02:44:38 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 02:44:33 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id XAA10926 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 23:00:15 -0800 Date: Mon, 26 Feb 2001 23:00:15 -0800 From: hal@finney.org Message-Id: <200102270700.XAA10926@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Scalability Adam writes: > - a system to find the URLs you want > - a system to locate a copy of and download the URL content > > Steve Schear and I were discussing the first one and another > approach is to just let a search engine do it for you. Either > by exposing the URLs to the real web through gateways which > make it appear (to search engines only perhaps) as static content. Web spidering does work to some extent, but not really that well. A substantial fraction of web pages aren't indexed. It should be possible to do better if the network were designed for it from the beginning. > Locating a copy of a target URL to download is a different > problem. Graydon's log map seems like a reasaonable approach. What is this? Is it similar to the OceanStore method of mapping a tree onto a network? I'm not familiar with this data structure. Hal From hal@finney.org Tue Feb 27 02:51:49 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 02:51:45 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id XAA11184 for bluesky@franklin.oit.unc.edu; Mon, 26 Feb 2001 23:49:01 -0800 Date: Mon, 26 Feb 2001 23:49:01 -0800 From: hal@finney.org Message-Id: <200102270749.XAA11184@finney.org> To: bluesky@franklin.oit.unc.edu Subject: p2p-hackers mailing list starting According to the news column at http://www.openp2p.com/, a new mailing list for coordination and communication between people working on various peer-to-peer software projects has been started. Here is the initial welcome message: : At the first O'Reilly Peer-to-Peer Conference in San Francisco in : February of 2001, I had the pleasure of meeting and talking to hackers : who were actively working on peer to peer systems. : : The first thing I learned is that we had different words for many : similar or identical concepts. Much of the early conversation was : simply trying to understand what the other was saying despite : different terminology. : : The second thing I learned (thanks especially to Wes Felter's : presentation), was that different p2p designers had made different : architectural decisions on a few key points, and had never looked back : after that early architectural decision. : : The third thing I learned (thanks especially to Branden Wiley and : Seguei Osokino, and Wes Felter again) is that interoperability between : different p2p file-sharing networks looks like an easier hack than I : would have guessed. : : So here is a mailing list which I hope will continue the noble : tradition of fraternization among p2p hackers. I would like to : continue the process we've begun of generating common terminology, a : common architectural taxonomy, and just "nuts and bolts" engineering : chit-chat about things that are important to all of us, for example : TCP vs. HTTP (vs. UDP?), MD5 vs. SHA1 (just don't use MD5!), bundled : meta-data vs. referenced meta-data, and probably a thousand other : important and interesting details. : : Mailing lists are living things, and I expect that I (or someone else) : will have to replace this welcome message as soon as we evolve away : from these initial ideas. : : Don Marti is the List Maintainer, and we are using his scheme of : requiring approved registration but approving everyone who asks. : : The initial invite list is a bunch of hackers who have specific : expertise and experience in p2p engineering, and who expressed : interest in this list. Feel free to invite others that you know who : can contribute to this list. : : Regards, Zooko From hal@finney.org Tue Feb 27 03:15:46 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 03:15:38 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id AAA11256 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 00:12:55 -0800 Date: Tue, 27 Feb 2001 00:12:55 -0800 From: hal@finney.org Message-Id: <200102270812.AAA11256@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: market-based data location (was Scalability) I don't have time to respond now to the many interesting points Wei raises in the message quoted below, so I'll just ask some questions: > The idea I'm playing with (and have been discussing with Ted Anderson > before creating this list) is for each data location server to self > select, based on market conditions, some criteria for what data to track, > for example any data located in a radius of x from itself, and has a > (content-hash based) GUID inside a radius of y from it's own GUID. It > would then accept paid or unpaid advertisement from data storage servers, > and answer paid or unpaid queries for data location. It can also query > other data location servers to fill its own database. Besides tracking > data, it would also track other location servers whose ranges overlap with > its own, so that if it can't answer a query directly it will return a list > of other location servers (along with their tracking and pricing policies) > who might be able to or at least get you closer to such a server, and > attest to their reliability and comprehensiveness. > > With this system you would see locations servers that are specialized for > different market niches. Some would be neighborhood trackers, with a small > network radius and a large GUID radius. Some would be global tracks, with > a large network radius and a small GUID radius. Some would be highly > priced and specialize in censored data. Some would acquire a reputation > for reliability and make money from long-term paid advertisements. Some > would have a low query price and make money from queries for popular data. > Some would only track other location servers and essentially provide a > reputation service. It would be really cool to watch this market > self-organize. To clarify, the data location service described here is the one which, given the GUID (content hash) of a piece of data, returns the address of a computer somewhere in the world which has that data? The question is, how would users find the right data location servers to query? How much have we reduced the problem in going from locating the node with the data, to locating the node which indexes the data? I can see that it is a step forward, because hopefully there would be fewer index nodes than data nodes. But even so, there could be far too many index nodes for end users to have lists of all of them. Do we need a higher level meta-index to describe the index nodes? (And it might not really move us forward that much. MojoNation uses a similar concept for its "trackers", but they expect almost everyone to be running trackers, I think. MN could have as many index nodes as file servers.) The idea of nodes specializing on the basis of content or popularity seems to make things even more complicated. Now it is no longer a purely mechanical process to find the right index node, one which depends solely on the GUID. We would need extra hints about the type of data being fetched to know where to begin searching among the many flavors and families of index nodes which exist. It's hard to see such mechanisms contributing to what one would hope would be a relatively mechanical process of fetching data. I need to study the OceanStore system more closely; I'll try to post something on it tomorrow. But I was struck by the statement in the introduction to one of their papers that their goal is to support a network with 10^10 users (10 billion users, i.e. the total population of Earth), each storing 10000 files, for a total of 100 trillion files. That's a lot of files! Scaling to that level will be a real challenge. Hal From weidai@eskimo.com Tue Feb 27 05:57:52 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 05:57:44 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id CAA14334 for ; Tue, 27 Feb 2001 02:57:39 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id CAA04165 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 02:57:38 -0800 (PST) Date: Tue, 27 Feb 2001 02:57:37 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: market-based data location (was Scalability) Message-ID: <20010227025737.A3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Tue, Feb 27, 2001 at 12:12:55AM -0800 On Tue, Feb 27, 2001 at 12:12:55AM -0800, hal@finney.org wrote: > To clarify, the data location service described here is the one which, > given the GUID (content hash) of a piece of data, returns the address > of a computer somewhere in the world which has that data? Yes. > The question is, how would users find the right data location servers > to query? How much have we reduced the problem in going from locating > the node with the data, to locating the node which indexes the data? > > I can see that it is a step forward, because hopefully there would be > fewer index nodes than data nodes. But even so, there could be far too > many index nodes for end users to have lists of all of them. Do we need > a higher level meta-index to describe the index nodes? In my scheme, location servers track each other as well as data objects. So if you start with one location server (which you obtain out of band) you can eventually find the location server that covers the object you're looking for. > (And it might not really move us forward that much. MojoNation uses > a similar concept for its "trackers", but they expect almost everyone > to be running trackers, I think. MN could have as many index nodes as > file servers.) Is there a paper that describes MojoNation in more detail than the technical overview? As far as I can tell from the technical overview[1], in MojoNation there is a central meta-tracker, which knows the address of every server in the network and what services they provide. And then there are content trackers and publication tracks which are distributed. My understanding is that content trackers provide keyword or metadata searching, and publication trackers are used by block servers to find and move data amongst themselves. So in order to find a data block from its content-hash ID, you first query the meta-tracker to obtain a block server whose block ID range covers your target. Then you query the block server, and it either gives you the block directly or finds it through publication trackers. Here the central meta-tracker is an obvious vulnerability and bottle neck. > The idea of nodes specializing on the basis of content or popularity > seems to make things even more complicated. Now it is no longer a purely > mechanical process to find the right index node, one which depends solely > on the GUID. We would need extra hints about the type of data being > fetched to know where to begin searching among the many flavors and > families of index nodes which exist. It's hard to see such mechanisms > contributing to what one would hope would be a relatively mechanical > process of fetching data. Hints are helpful but not necessary. A typical strategy for the user agent would be to query the local-area location server first, then a server that covers a larger network area, then a global server that specializes in the target GUID range, and finally an overseas server that specializes in censored data. > I need to study the OceanStore system more closely; I'll try to post > something on it tomorrow. But I was struck by the statement in the > introduction to one of their papers that their goal is to support a > network with 10^10 users (10 billion users, i.e. the total population > of Earth), each storing 10000 files, for a total of 100 trillion files. > That's a lot of files! Scaling to that level will be a real challenge. Only 10000 files per user? On my hard drive I have directories that contain more than 10000 files each. To be completely general purpose I think you need to aim for at least 10^6 files per user. [1] http://www.mojonation.net/docs/technical_overview.shtml From weidai@eskimo.com Tue Feb 27 06:54:04 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 06:53:59 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id DAA28897 for ; Tue, 27 Feb 2001 03:53:54 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id DAA10589 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 03:53:53 -0800 (PST) Date: Tue, 27 Feb 2001 03:53:53 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227035353.D3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from ian@octayne.com on Mon, Feb 26, 2001 at 03:03:10PM -0800 On Mon, Feb 26, 2001 at 03:03:10PM -0800, Ian Clarke wrote: > There has also been commentary on the fact that Freenet drops unpopular > files. This is a frequent topic of discussion on Freenet's mailing > lists too, but we long-ago agreed that the purpose of Freenet was never > to be a robust file repository, any more than broadcasting something > over the radio is a reliable means to ensure it will be recorded for > posterity. I think the main question here is whether eventually there is room for more than one global-scale distributed storage system. Suppose there is a system that provides all of the benefits that Freenet does, but also has controllable reliability (so that one can pay more to obtain greater reliability). Then it's hard to see how Freenet can compete with it unless Freenet has become so popular that no one wants to switch unless everyone switches. > I notice that people frequently neglect to bear-in-mind the goals of a > system when criticising features of it. If Freenet didn't drop files > then it could eventually become full, which would be unacceptable, and > would provide a trivial attack against it. A MojoNation-style economy > is not acceptable either as we believe that people should be able to > publish information regardless of their personal wealth. There are > other reasons too. How does Freenet prevent an attacker from inserting so much garbage that most useful files are dropped, or people from using it to trade huge files without contributing resources back to the system? In a MojoNation-style economy anyone can also publish information regardless of their personal wealth. This is because storage and bandwidth are very cheap and getting cheaper. Consider the fact that there are a lot of unused disk space and bandwidth sitting on everyone's desks. If they all partcipate in the market it will be very inexpensive to publish anything except things like high resolution motion pictures. From hal@finney.org Tue Feb 27 12:23:05 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 12:23:01 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id JAA11998 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 09:20:16 -0800 Date: Tue, 27 Feb 2001 09:20:16 -0800 From: hal@finney.org Message-Id: <200102271720.JAA11998@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: market-based data location (was Scalability) Wei writes: > Is there a paper that describes MojoNation in more detail than the > technical overview? As far as I can tell from the technical overview[1], > in MojoNation there is a central meta-tracker, which knows the address of > every server in the network and what services they provide. And then there > are content trackers and publication tracks which are distributed. My > understanding is that content trackers provide keyword or metadata > searching, and publication trackers are used by block servers to find and > move data amongst themselves. So in order to find a data block from its > content-hash ID, you first query the meta-tracker to obtain a block server > whose block ID range covers your target. Then you query the block server, > and it either gives you the block directly or finds it through publication > trackers. Here the central meta-tracker is an obvious vulnerability and > bottle neck. There is a file describing their plans for distributed metatracking at http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/~checkout~/evil/hackerdocs/distributed_metatracking.html?cvsroot=mojonation However I am somewhat worried by: This idea is simply "every broker a metatracker". I think we can use the current metatracker protocols and the current source code to deploy really good distributed metatracking. I'm not sure it really moves us forward much if every node stores data, every node runs trackers to index data, and now every node will run a metatracker to index the trackers. It's starting to sound like "turtles all the way down". Hal From ian@technic.uprizer.com Tue Feb 27 13:48:17 2001 Return-Path: Received: from technic.uprizer.com ([4.18.49.34]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 13:48:13 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f1RIswY02098 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 10:54:58 -0800 Date: Tue, 27 Feb 2001 10:54:58 -0800 From: Ian Clarke To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227105458.G1331@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="7uYPyRQQ5N0D02nI" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from weidai@eskimo.com on Tue, Feb 27, 2001 at 03:53:53AM -0800 --7uYPyRQQ5N0D02nI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 27, 2001 at 03:53:53AM -0800, Wei Dai wrote: > I think the main question here is whether eventually there is room for > more than one global-scale distributed storage system. Suppose there is a > system that provides all of the benefits that Freenet does, but also has > controllable reliability (so that one can pay more to obtain greater > reliability). Then it's hard to see how Freenet can compete with it unless > Freenet has become so popular that no one wants to switch unless everyone > switches. I suspect that the two may be mutually exclusive, although don't ask me to prove it ;) In Freenet, information's availability is in proportion to its demand. This is probably the perfect situation from the perspective of consumers of information, but not for producers of information (I had this argument with Richard Stallman over a year ago). If you take television as an analogy, some of the information is there because it is popular (the content), and some is there because people have paid to put it there (the advertisments). From the point of view of someone watching TV - would you prefer a system with or without advertisments? > How does Freenet prevent an attacker from inserting so much garbage that > most useful files are dropped This would be difficult but is possible (in a Freenet which was being widely used, any spamming would be a drop in the ocean), we will be taking additional precautions which will make this still more difficult (inserted data is actually inserted half-way up the datastore rather than at the top). > or people from using it to trade huge files > without contributing resources back to the system? The huge file *is* a resource, but its popularity (in proportion to its size) will determine the degree to which it is replicated on Freenet, if at all. You seem to be thinking in terms of providing a service to producers of information, where as Freenet provides a service to consumers of information. From this point of view, all information is a resource (the value of which is determined by popularity). > In a MojoNation-style economy anyone can also publish information > regardless of their personal wealth. This is because storage and bandwidth > are very cheap and getting cheaper. Consider the fact that there are a lot > of unused disk space and bandwidth sitting on everyone's desks. If they > all partcipate in the market it will be very inexpensive to publish > anything except things like high resolution motion pictures. Most people in the world today have 28.8k or 33.6k modems - and pay for internet connectivity by the minute, that is hardly what I would describe as a surplus of bandwidth. Ian. --7uYPyRQQ5N0D02nI Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6m/gCQtgxRWSmsqwRAgnvAJ9ew/kl8QaAwaThYchsYdytbwHGgwCfWf7L DwUFdOPGzBSui9/GlD9oWEg= =nG8d -----END PGP SIGNATURE----- --7uYPyRQQ5N0D02nI-- From hal@finney.org Tue Feb 27 18:51:13 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 18:51:04 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id PAA13401 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 15:48:20 -0800 Date: Tue, 27 Feb 2001 15:48:20 -0800 From: hal@finney.org Message-Id: <200102272348.PAA13401@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Hi, and some comments Wei writes: > How does Freenet prevent an attacker from inserting so much garbage that > most useful files are dropped, or people from using it to trade huge files > without contributing resources back to the system? > > In a MojoNation-style economy anyone can also publish information > regardless of their personal wealth. This is because storage and bandwidth > are very cheap and getting cheaper. Consider the fact that there are a lot > of unused disk space and bandwidth sitting on everyone's desks. If they > all partcipate in the market it will be very inexpensive to publish > anything except things like high resolution motion pictures. One problem which MojoNation is running into is that people don't want to have to pay to publish. Most people, when they first hear about MN, understand right away the idea of paying mojo to download files. However they assume that this means that they will get paid to supply popular files for download. This is seen as a way to discourage "leeching" and to encourage people to make material available, which has been a problem on networks like Gnutella. When they discover that they have to pay to put material out into the network for downloading by others, they become discouraged about the prospects for MN to succeed in today's crowded P2P environment. Successfully launching a P2P system requires providing material that people want to download. And part of this success has to mean making it easy for people to put stuff out there. By putting barriers in the way of publishing, MN is setting itself up with a huge handicap. To some extent they have compensated for it at present by letting everyone have as much Mojo as they want upon request, ten million at a time. This totally undercuts the economic foundation for their model and presumably they intend to stop these giveaways eventually. But the fact that they've had to do this is an indication that "pay to publish" just won't fly. Hal From weidai@eskimo.com Tue Feb 27 20:06:49 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:06:45 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id RAA06848 for ; Tue, 27 Feb 2001 17:06:36 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id RAA20404 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 17:06:36 -0800 (PST) Date: Tue, 27 Feb 2001 17:06:35 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227170635.J3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Tue, Feb 27, 2001 at 03:48:20PM -0800 On Tue, Feb 27, 2001 at 03:48:20PM -0800, hal@finney.org wrote: > One problem which MojoNation is running into is that people don't want to > have to pay to publish. > > Most people, when they first hear about MN, understand right away the > idea of paying mojo to download files. However they assume that this > means that they will get paid to supply popular files for download. > This is seen as a way to discourage "leeching" and to encourage people > to make material available, which has been a problem on networks like > Gnutella. Perhaps the problem is that MojoNation is not flexible enough. If each storage servers can decide for itself exactly what data to store (rather than a block ID range as in MojoNation), each person can publish on his own server, like in Napster, and just make money when other people download. Paying someone else to publish should be necessary only if you want better reliability (i.e., long term offsite storage). From pete@retep.yi.org Tue Feb 27 20:26:09 2001 Return-Path: Received: from femail2.rdc1.on.home.com ([24.2.9.89]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:26:06 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail2.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010228012547.NZTF606.femail2.rdc1.on.home.com@gw.localdomain>; Tue, 27 Feb 2001 17:25:47 -0800 Received: from pete by gw.localdomain with local (Exim 3.16 #1 (Debian)) id 14XvMG-0003pl-00; Tue, 27 Feb 2001 20:24:48 -0500 Date: Tue, 27 Feb 2001 20:24:48 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems , devel@freenetproject.org Subject: Re: Hi, and some comments Message-ID: <20010227202448.E14374@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="qOrJKOH36bD5yhNe" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from weidai@eskimo.com on Tue, Feb 27, 2001 at 03:53:53AM -0800 Sender: Peter Todd --qOrJKOH36bD5yhNe Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 27, 2001 at 03:53:53AM -0800, Wei Dai wrote: > > I notice that people frequently neglect to bear-in-mind the goals of a > > system when criticising features of it. If Freenet didn't drop files > > then it could eventually become full, which would be unacceptable, and > > would provide a trivial attack against it. A MojoNation-style economy > > is not acceptable either as we believe that people should be able to > > publish information regardless of their personal wealth. There are > > other reasons too. >=20 > How does Freenet prevent an attacker from inserting so much garbage that > most useful files are dropped, or people from using it to trade huge files > without contributing resources back to the system? It doesn't. A "drop piles of data onto one node and request it from another" attack could do some real damage. An insertion is going to take a maximum of 100 times more bandwidth then the attacker uses. (max hops is 100) On request it would probably be closer to 5-15 times more bandwidth depending on how well the routing is working. =20 However any such attack could be politically infeasable if Freenet got popular enough. The amount of data being transfered around the internet as a whole would cripple the internet itself. Currently the freenet routing doesn't take internet-level closeness into account so the real bandwidth consumed by Freenet is nodes * requests per second * average number of freenet hopes * average number of real hops. Under large-scale attack that would be a staggering amount of bandwidth. --=20 retep@penguinpowered.com http://retep.tripod.com=20 --qOrJKOH36bD5yhNe Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6nFNgLdZCJKx+dn0RAtb+AJ9vZVuMM/KSq9uEhlX7Ny9J1eA13ACfablp d3raE/JKPaoLqRXPVxDDYm0= =FdM6 -----END PGP SIGNATURE----- --qOrJKOH36bD5yhNe-- From ian@technic.uprizer.com Tue Feb 27 20:43:02 2001 Return-Path: Received: from technic.uprizer.com ([4.18.49.34]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:42:58 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f1S1nhe07659 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 17:49:43 -0800 Date: Tue, 27 Feb 2001 17:49:43 -0800 From: Ian Clarke To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227174943.B1331@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="7A15IXT36zwL5BDA" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from weidai@eskimo.com on Tue, Feb 27, 2001 at 05:06:35PM -0800 --7A15IXT36zwL5BDA Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Feb 27, 2001 at 05:06:35PM -0800, Wei Dai wrote: > Perhaps the problem is that MojoNation is not flexible enough. If each > storage servers can decide for itself exactly what data to store (rather > than a block ID range as in MojoNation), each person can publish on his > own server, like in Napster, and just make money when other people > download. Paying someone else to publish should be necessary only if you > want better reliability (i.e., long term offsite storage). You view the network as providing a service to publishers. The alternative is Freenet's perspective which is that publishers are providing a service to consumers of information, and thus we *encourage* people to publish by not charging them, instead consumers of information provide disk space in return for the content. Even still, getting people to publish on Freenet is a real challenge, and this would be even more of a challenge for MN for the reasons Hal outlined. Ian. --7A15IXT36zwL5BDA Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6nFk3QtgxRWSmsqwRAoVrAJ9sWktal7GYTr7+Kiz1PPLT0xNWBQCeNZsi o/HG7eLQuTttxUWMP62OcCY= =oMVt -----END PGP SIGNATURE----- --7A15IXT36zwL5BDA-- From blanu@uts.cc.utexas.edu Tue Feb 27 20:46:38 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:46:33 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id TAA11786 for ; Tue, 27 Feb 2001 19:46:27 -0600 (CST) Date: Tue, 27 Feb 2001 19:46:27 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > I think the main question here is whether eventually there is room for > more than one global-scale distributed storage system. Suppose there is a > system that provides all of the benefits that Freenet does, but also has > controllable reliability (so that one can pay more to obtain greater > reliability). Then it's hard to see how Freenet can compete with it unless > Freenet has become so popular that no one wants to switch unless everyone > switches. There is definitely room for more than one global-scale distributed storage system because not everyone has the same goals for such a system. Since it is probably not possible to optimize for all goals, there are different (not competing) models that can coexist. Speed and anonymity, for instance, are usually opposed when tradeoffs are being made. An example of this is mix networks which are pretty much indispensible for defeating traffic analysis but definitionally make network traffic slower than without them. There are also different trust models and differente resource/load balancing models. Some tradeoffs may be able to be overcome. The Freenet developers currently suspect that we may be able to make Freenet reliable without throwing away anonymity, thus briding the distribution/storage network gap. But we'll see how that turns out. Anyway, even if some differentiations can be broken down, I think that the speed/anonymity one will prove to be pretty sticky. So the question of what if a network which does everything that Freenet does and more come around is a moot question due to design tradeoffs. BTW, I wrote about this in the Gateways chapter of the O'Reilly P2P book if anyone happens to have it. > In a MojoNation-style economy anyone can also publish information > regardless of their personal wealth. This is because storage and bandwidth > are very cheap and getting cheaper. Consider the fact that there are a lot > of unused disk space and bandwidth sitting on everyone's desks. If they > all partcipate in the market it will be very inexpensive to publish > anything except things like high resolution motion pictures. This is only one situation that exists in the world. Freenet also seeks to serve another situation that exists in the world in which people have scarce computing resources. It is particularly important to provide access for people with transient access that can't run permanent nodes. The ability to dialup, access/upload information, and then disconnect and run away is important when you're trying to facilitate illicit communication among oppressed groups. From blanu@uts.cc.utexas.edu Tue Feb 27 20:49:06 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:49:01 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id TAA02446 for ; Tue, 27 Feb 2001 19:48:55 -0600 (CST) Date: Tue, 27 Feb 2001 19:48:55 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > One problem which MojoNation is running into is that people don't want to > have to pay to publish. I've been playing with MojoNation a lot lately and this was my first thought of something that needs to be changed. When the success of the network depends on its content, charging for publication is bad. On the other hand, eliminating spam is good. So what you really want is a situation in which you are charged to publish, but if you publish popular content then you make a return on your investment. I remember hearing about a tip button so that you can tip the publisher, but I haven't actually seen this in the UI. From weidai@eskimo.com Tue Feb 27 21:00:05 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 20:59:58 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id RAA30863 for ; Tue, 27 Feb 2001 17:59:51 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id RAA23849 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 17:59:49 -0800 (PST) Date: Tue, 27 Feb 2001 17:59:49 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227175947.K3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from ian@octayne.com on Tue, Feb 27, 2001 at 10:54:58AM -0800 On Tue, Feb 27, 2001 at 10:54:58AM -0800, Ian Clarke wrote: > I suspect that the two may be mutually exclusive, although don't ask me > to prove it ;) In Freenet, information's availability is in proportion > to its demand. This is probably the perfect situation from the > perspective of consumers of information, but not for producers of > information (I had this argument with Richard Stallman over a year ago). But nobody consumes *just* popular data, so you still need a system that can handle the less popular data. What are your thoughts on OceanStore? It has problems, but I think it clearly proves that it is possible to handle both popular and unpopular data in the same system. Do you have a good idea of how availability scales with popularity in Freenet? For example, from what percentage of the network would you be able to find a piece of data that has been requested by 50% of the nodes in the last month? 10%, 1%, .1%, etc.? What kind of goals are you aiming for in this regard? > If you take television as an analogy, some of the information is there > because it is popular (the content), and some is there because people > have paid to put it there (the advertisments). From the point of view > of someone watching TV - would you prefer a system with or without > advertisments? As a TV watcher, I obviously prefer a system with ads, because otherwise there wouldn't be any professional free television programs. But that's a bad analogy; here the problem is how to ensure less popular data can get to people who specificly request them, not how to fund original content. > The huge file *is* a resource, but its popularity (in proportion to its > size) will determine the degree to which it is replicated on Freenet, if > at all. You seem to be thinking in terms of providing a service to > producers of information, where as Freenet provides a service to > consumers of information. From this point of view, all information is a > resource (the value of which is determined by popularity). By resources I meant storage space and bandwidth, rather than content. Who will be paying for them in Freenet? Who will be paying for content is a whole other issue which probably deserves a seperate mailing list to talk about. > Most people in the world today have 28.8k or 33.6k modems - and pay for > internet connectivity by the minute, that is hardly what I would > describe as a surplus of bandwidth. Those people obviously will not be participating as storage servers. But there are a lot of people with broadband Internet connections, and they will provide sufficient supply to ensure very low publication costs. From md98-osa@nada.kth.se Tue Feb 27 21:00:26 2001 Return-Path: Received: from mail1.telia.com ([62.20.149.43]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 21:00:14 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14Xvwd-0000UW-00 for bluesky@franklin.oit.unc.edu; Wed, 28 Feb 2001 03:02:23 +0100 Date: Wed, 28 Feb 2001 03:02:23 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: market-based data location (was Scalability) Message-ID: <20010228030223.B1131@hobbex.localdomain> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Tue, Feb 27, 2001 at 09:20:16AM -0800 On Tue, Feb 27, 2001 at 09:20:16AM -0800, hal@finney.org wrote: > Wei writes: <> > There is a file describing their plans for distributed metatracking at > http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/~checkout~/evil/hackerdocs/distributed_metatracking.html?cvsroot=mojonation > > However I am somewhat worried by: > > This idea is simply "every broker a metatracker". I think we can use > the current metatracker protocols and the current source code to deploy > really good distributed metatracking. > > I'm not sure it really moves us forward much if every node stores data, > every node runs trackers to index data, and now every node will run a > metatracker to index the trackers. It's starting to sound like "turtles > all the way down". I approached the Mojonation people regarding my concern that the whole content tracking system was not really decentralized at all at the Peer to Peer conference. Even if this system works in other respects, with n documents on the network, you need to satisfy: * = n Since both these numbers need to be kept down, the best result would be sqrt(n) for both, but sqrt(n) not really good enough for me to consider the network decentralized (in the sense of many documents not depending on few nodes) or scalable (in the sense of each user request not taking unreasonable work). It turned out however, that Mojonation's basic architecture isn't scalable anyways. The content-trackers in MN do not map metadata onto a physical address, they map metadata onto UIDs (the SHA1 hash for each part) of the data. Apparently, to then find the data, MN simply expects each node to keep a list of the IDs of every other node in the network, and then compares the least order bytes of the node IDs with the data UID to find the node where to put or look for a piece of data. With this is mind, complaining about the scalability of the other aspects of their network seems a little silly. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From hal@finney.org Tue Feb 27 21:56:16 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 21:56:11 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id SAA14094 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 18:53:26 -0800 Date: Tue, 27 Feb 2001 18:53:26 -0800 From: hal@finney.org Message-Id: <200102280253.SAA14094@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Hi, and some comments Wei writes: > On Tue, Feb 27, 2001 at 10:54:58AM -0800, Ian Clarke wrote: > > The huge file *is* a resource, but its popularity (in proportion to its > > size) will determine the degree to which it is replicated on Freenet, if > > at all. You seem to be thinking in terms of providing a service to > > producers of information, where as Freenet provides a service to > > consumers of information. From this point of view, all information is a > > resource (the value of which is determined by popularity). > > By resources I meant storage space and bandwidth, rather than content. Who > will be paying for them in Freenet? Who will be paying for content is a > whole other issue which probably deserves a seperate mailing list to talk > about. > > > Most people in the world today have 28.8k or 33.6k modems - and pay for > > internet connectivity by the minute, that is hardly what I would > > describe as a surplus of bandwidth. > > Those people obviously will not be participating as storage servers. But > there are a lot of people with broadband Internet connections, and they > will provide sufficient supply to ensure very low publication costs. I think there are two uses for money being discussed here: one is to repay those who are offering storage space and bandwidth for their time and expense; and the other is to "keep score" and provide a mechanism for deciding which data to keep and which to drop. If storage and bandwidth are in sufficient supply that publication costs will be very low, do we really have to worry that no one will pay for them in Freenet? Isn't it likely that people will simply offer them for free, out of altruistic reasons, since it costs them very little, just as they now leave Napster running? I don't think that Freenet will necessarily lack for storage space and bandwidth even though people aren't paid for them. As for the score-keeping role of money, Freenet proposes to use direct measures of popularity. While these can be manipulated, if the number of people who want to do so is a small percentage, the goal is that the network as a whole will not be too much affected. (It would be good to see some calculations along these lines, though.) BTW OpenP2P has an article challenging the notion that charging for resources a la MojoNation makes sense, Clay Shirky's "In Praise of Freeloading" [1], with a response by MN's Jim McCoy [2]. Hal [1] http://www.openp2p.com/pub/a/p2p/2000/12/01/shirky_freeloading.html [2] http://www.openp2p.com/pub/a/p2p/2001/01/11/mojo.html From weidai@eskimo.com Tue Feb 27 22:32:04 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 22:31:58 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id TAA17779 for ; Tue, 27 Feb 2001 19:31:27 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id TAA01414 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 19:31:27 -0800 (PST) Date: Tue, 27 Feb 2001 19:31:27 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227193127.L3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from blanu@uts.cc.utexas.edu on Tue, Feb 27, 2001 at 07:46:27PM -0600 On Tue, Feb 27, 2001 at 07:46:27PM -0600, Brandon wrote: > So the question of what if a network which does everything that Freenet > does and more come around is a moot question due to design tradeoffs. But we should find out which tradeoffs are necessary and which ones are not. My concern is that a "good-enough" system will get a monopoly and it will be impossible to overhaul its architecture. Just because different projects have different goals it doesn't mean they'll all survive or be used. I think it's quite likely that the network effect will be so strong that the first system to become popular will have a monopoly on this entire space. And if for example it does not offer censorship resistance or reliability, it may become impossible to create a viable second system that does. > This is only one situation that exists in the world. Freenet also seeks to > serve another situation that exists in the world in which people have > scarce computing resources. It is particularly important to provide access > for people with transient access that can't run permanent nodes. The > ability to dialup, access/upload information, and then disconnect and run > away is important when you're trying to facilitate illicit communication > among oppressed groups. This scenario is completely compatible with what I wrote earlier. In a market-based system, the people who have storage and bandwidth will rent it to those who don't. The prices will certainly be low enough that the costs for anything you can upload through a dialup link will be negligible. And also there will probably be plenty of servers willing to donate storage to worthy causes. Keep in mind that the purpose of having payment is to control abuses and defend against attacks, and to allow people who contribute resources to recover their costs. It may be difficult or impossible to do these things without it. From blanu@uts.cc.utexas.edu Tue Feb 27 22:47:38 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 22:47:34 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id VAA09074 for ; Tue, 27 Feb 2001 21:47:28 -0600 (CST) Date: Tue, 27 Feb 2001 21:47:27 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > But we should find out which tradeoffs are necessary and which ones are > not. My concern is that a "good-enough" system will get a monopoly and it > will be impossible to overhaul its architecture. Indeed. > Just because different > projects have different goals it doesn't mean they'll all survive or be > used. Agreed. > I think it's quite likely that the network effect will be so strong > that the first system to become popular will have a monopoly on this > entire space. And if for example it does not offer censorship resistance > or reliability, it may become impossible to create a viable second system > that does. I don't think so. If it doesn't offer censorship resistance then it obviously can't be used by anyone that needs it. If it doesn't offer reliability then it can't be used by anyone that needs that. I think that the biggest risk is for "optional but nice" features like micropayments and quality of service assurances being left out. > This scenario is completely compatible with what I wrote earlier. In a > market-based system, the people who have storage and bandwidth will rent > it to those who don't. The prices will certainly be low enough that the > costs for anything you can upload through a dialup link will be > negligible. And also there will probably be plenty of servers willing to > donate storage to worthy causes. If you can't run a node then how do you collect capital to pay these low fees? > Keep in mind that the purpose of having payment is to control abuses and > defend against attacks, and to allow people who contribute resources to > recover their costs. It may be difficult or impossible to do these things > without it. As far as defending against attacks, micropayments are one way and it's a nice way. There are other ways as well which might work. Recover costs is a different issue and I'm pretty happy with a voluntary network in which the cost you pay to be a part of the network (ISP and bandwidth charges) is the cost you pay to be able to access the network's resources. But this is of course not a suitable model for everyone's needs. Some people are going to want more accounting and reciprocity of resource usage. For Freenet, however, where the goal is anonymity, accounting is exactly what we don't want. I would, however, be interested in hearing about any sort of system of symbolic reciprocity balancing (micropayments) which is totally decentralized. Ever system that I'm aware of uses a central trust authority (bank), which is the main reason that I consider micropayments to be unsuitable for Freenet. Obviously some system of abuse management is needed, but it has to be fully decentralized for our particular needs. From mfreed@MIT.EDU Tue Feb 27 23:08:15 2001 Return-Path: Received: from fort-point-station.mit.edu ([18.72.0.53]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 23:08:11 -0500 Received: from central-city-carrier-station.mit.edu (CENTRAL-CITY-CARRIER-STATION.MIT.EDU [18.7.21.75]) by fort-point-station.mit.edu (8.9.2/8.9.2) with ESMTP id XAA11598 for ; Tue, 27 Feb 2001 23:08:05 -0500 (EST) Received: from melbourne-city-street.MIT.EDU (MELBOURNE-CITY-STREET.MIT.EDU [18.69.0.45]) by central-city-carrier-station.mit.edu (8.9.2/8.9.2) with ESMTP id XAA04562 for ; Tue, 27 Feb 2001 23:05:01 -0500 (EST) Received: from griffen (GRIFFEN.MIT.EDU [18.239.2.119]) by melbourne-city-street.MIT.EDU (8.9.3/8.9.2) with SMTP id XAA10824 for ; Tue, 27 Feb 2001 23:05:00 -0500 (EST) Message-Id: <200102280405.XAA10824@melbourne-city-street.MIT.EDU> X-Sender: mfreed@hesiod X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.2 Date: Tue, 27 Feb 2001 23:04:10 -0500 To: "Global-Scale Distributed Storage Systems" From: Michael J Freedman Subject: Re: Hi, and some comments In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 09:47 PM 2/27/2001 -0600, you wrote: >For Freenet, however, where the goal is anonymity, accounting is exactly >what we don't want. I would, however, be interested in hearing about any >sort of system of symbolic reciprocity balancing (micropayments) which is >totally decentralized. Ever system that I'm aware of uses a central trust >authority (bank), which is the main reason that I consider micropayments >to be unsuitable for Freenet. Obviously some system of abuse management is >needed, but it has to be fully decentralized for our particular needs. The requirement for some centralized bank is only necessary when you are describing some fungible micropayment scheme -- that is, payments are redeemable for other services in your system. If your concern is largely to prevent abuses -- e.g., congestion management for temporary resources -- you can use non-fungible type schemes, such as proofs of work (PoW). Adam Back's hash cash is a clear example of such. These don't require any centralized bank server...coins are "minted" recipient-specific. (I think Wei Dei's B-money protocol also describes a decentralized system, although of a pseudo-reputation-based nature.) We describe this whole issue of accountability with (pseudo) anonymity in the O'Reilly P2P book. You can find a copy of our accountability chapter at: http://freehaven.net/papers.html Currently, we're rethinking the FreeHaven design. One of our plans is to develop a free, low-latency, two-way mix-net for an anonymizing layer. We hope to be able to use such as a pluggable research tool to study micropayment, reputation, and other congestion management (abuse prevention) techniques. Thanks! --mike ----- "Not all those who wander are lost." mfreed@mit.edu From blanu@uts.cc.utexas.edu Tue Feb 27 23:19:42 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 23:19:38 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id WAA16230 for ; Tue, 27 Feb 2001 22:19:32 -0600 (CST) Date: Tue, 27 Feb 2001 22:19:32 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > you can use non-fungible type schemes, such as proofs of work (PoW). Adam > Back's hash cash is a clear example of such. I should have more clearly specified that I consider hash cash schemes to also be inappropriate as it then becomes an economic question of how much computing power you own. When the enemies of the system generally have more resources than the individual participants of the system, PoW schemes don't seem viable to me. So the micropayment scheme has to be one in which credit is given for participating in the network. Our have an important point in that such a scheme is inextricably tied to a reputation system. This isn't obvious with a centralized bank since you then have a centralized trust authority. But with a decentralized system you either have to have everyone issue person specific credits (you give me some stuff and then I'll give you some stuff) or has a way of determing whose currency is valid and whose is bunk. > Currently, we're rethinking the FreeHaven design. One of our plans is to > develop a free, low-latency, two-way mix-net for an anonymizing layer. We > hope to be able to use such as a pluggable research tool to study > micropayment, reputation, and other congestion management (abuse > prevention) techniques. I consider a free, low-latency, two-way mix network to be one of the most important things lacking in the P2P world right now. From mfreed@MIT.EDU Tue Feb 27 23:26:48 2001 Return-Path: Received: from pacific-carrier-annex.mit.edu ([18.7.21.83]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 23:26:45 -0500 Received: from grand-central-station.mit.edu (GRAND-CENTRAL-STATION.MIT.EDU [18.7.21.82]) by pacific-carrier-annex.mit.edu (8.9.2/8.9.2) with ESMTP id XAA02170 for ; Tue, 27 Feb 2001 23:26:40 -0500 (EST) Received: from melbourne-city-street.MIT.EDU (MELBOURNE-CITY-STREET.MIT.EDU [18.69.0.45]) by grand-central-station.mit.edu (8.9.2/8.9.2) with ESMTP id XAA13242 for ; Tue, 27 Feb 2001 23:26:39 -0500 (EST) Received: from griffen (GRIFFEN.MIT.EDU [18.239.2.119]) by melbourne-city-street.MIT.EDU (8.9.3/8.9.2) with SMTP id XAA14208 for ; Tue, 27 Feb 2001 23:26:39 -0500 (EST) Message-Id: <200102280426.XAA14208@melbourne-city-street.MIT.EDU> X-Sender: mfreed@hesiod X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.2 Date: Tue, 27 Feb 2001 23:25:56 -0500 To: "Global-Scale Distributed Storage Systems" From: Michael J Freedman Subject: Re: Hi, and some comments In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 10:19 PM 2/27/2001 -0600, you wrote: > >> you can use non-fungible type schemes, such as proofs of work (PoW). Adam >> Back's hash cash is a clear example of such. > >I should have more clearly specified that I consider hash cash schemes to >also be inappropriate as it then becomes an economic question of how much >computing power you own. When the enemies of the system generally have >more resources than the individual participants of the system, PoW schemes >don't seem viable to me. So the micropayment scheme has to be one in which >credit is given for participating in the network. Well, this becomes one of resource allocation techniques. If the system is not currently loaded, micropayments may not be necessary. (Our claim was that payments are always necessary *only* when resources are used up cumulatively -- such as permanent storage -- alternatively, for temporarily-allocated resources (cache, bandwidth, etc.), payments may only be necessary when the system is congested.) But anyway, the above argument holds when a adversary can capitalize greater resources to defeat the amount of work necessary to be shown. There is an alternative, although more theoretical, for a class of computationally difficult problems shown to be non-parallelizable (\element NC). This was the basis for the LCS35 time capsule crypto puzzle. Anyway, just thought I'd mention, --mike ----- "Not all those who wander are lost." mfreed@mit.edu From weidai@eskimo.com Tue Feb 27 23:30:13 2001 Return-Path: Received: from mx1.eskimo.com ([204.122.16.48]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 23:30:06 -0500 Received: from eskimo.com (weidai@eskimo.com [204.122.16.13]) by mx1.eskimo.com (8.9.1a/8.8.8) with ESMTP id UAA17263 for ; Tue, 27 Feb 2001 20:29:54 -0800 Received: (from weidai@localhost) by eskimo.com (8.9.1a/8.9.1) id UAA05222 for bluesky@franklin.oit.unc.edu; Tue, 27 Feb 2001 20:29:54 -0800 (PST) Date: Tue, 27 Feb 2001 20:29:54 -0800 From: Wei Dai To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010227202953.M3372@eskimo.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 1.0i In-Reply-To: ; from hal@finney.org on Tue, Feb 27, 2001 at 06:53:26PM -0800 On Tue, Feb 27, 2001 at 06:53:26PM -0800, hal@finney.org wrote: > If storage and bandwidth are in sufficient supply that publication costs > will be very low, do we really have to worry that no one will pay for > them in Freenet? Isn't it likely that people will simply offer them > for free, out of altruistic reasons, since it costs them very little, > just as they now leave Napster running? I don't think that Freenet > will necessarily lack for storage space and bandwidth even though people > aren't paid for them. That may be the case now, but are you sure that in the future the supply of freely donated storage and bandwidth will always be sufficient to meet demand? Without payment there is no easy way to attract more resources into the system. > As for the score-keeping role of money, Freenet proposes to use direct > measures of popularity. While these can be manipulated, if the number > of people who want to do so is a small percentage, the goal is that the > network as a whole will not be too much affected. (It would be good to > see some calculations along these lines, though.) Popularity is only one reason to store data. Some data may not be popular but may have high value for a small group of people or have long-term archival value. I guess Freenet focuses on popularity because it's harder to manipulate than subjective value, but the downside is that it can't handle the latter kind of data. A market-based system can automatically and optimally balance between the two. From blanu@uts.cc.utexas.edu Tue Feb 27 23:43:17 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Feb 2001 23:43:13 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id WAA30285 for ; Tue, 27 Feb 2001 22:43:05 -0600 (CST) Date: Tue, 27 Feb 2001 22:43:04 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > Popularity is only one reason to store data. Some data may not be popular > but may have high value for a small group of people or have long-term > archival value. I guess Freenet focuses on popularity because it's harder > to manipulate than subjective value, but the downside is that it can't > handle the latter kind of data. A market-based system can automatically > and optimally balance between the two. Popularity is local, not global, in Freenet. So if your small group of people with shared values have nodes which shared proximity then they will serve as an island in which that information will survive. It will disappear from the greater network as is proper since no one else wants it. Unfortunately, Freenet has been moving against my protestations towards a system of organization in which you cannot control the proximity of nodes to each other. In the future Freenet the random connections between nodes will mean that popularity is more or less global since you will be statistically likely to share proximity with plebian nodes rather than special interest nodes. I think that's too bad, but oh well. I think that ad-hoc clustering of nodes with shared intersts is a fascinating area of P2P development and would really like to hear about anyone that is working on this. The Freenet developers are exploring a system in which you can voluntarily give resources to the network in order to host data which you think has long-term archival value. This is still preliminary, though, and we don't know how it will work out. There are some issues with anonymity if you decide to host long-term data. We consider that to be okay since it's really just an option on top of our more anonymous system which doesn't supply long-term hosting. Hopefully the FreeHaven folks will come up with a more sophisticated system of anonymous long-term hosting which we can try to use. If not then that's okay because we fancy ourselves a distribution network anyway. I'm still interesting in gatewaying distribution and storage networks together, but there are of course obstacles in that direction as well. From twh1@longitude.doc.ic.ac.uk Wed Feb 28 08:55:20 2001 Return-Path: Received: from diver.doc.ic.ac.uk ([146.169.1.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 08:55:15 -0500 Received: from longitude.doc.ic.ac.uk ([146.169.1.26]) by diver.doc.ic.ac.uk with esmtp (Exim 3.16 #7) id 14Y74O-0008Pg-00 for bluesky@franklin.oit.unc.edu; Wed, 28 Feb 2001 13:55:08 +0000 Received: (from twh1@localhost) by longitude.doc.ic.ac.uk (8.8.7/8.8.8) id NAA01777 for bluesky@franklin.oit.unc.edu; Wed, 28 Feb 2001 13:55:45 GMT From: Theodore Hong Message-Id: <200102281355.NAA01777@longitude.doc.ic.ac.uk> Subject: Re: Hi, and some comments To: bluesky@franklin.oit.unc.edu Date: Wed, 28 Feb 2001 13:55:45 +0000 (GMT) In-Reply-To: from "hal@finney.org" at Feb 27, 2001 03:48:20 PM Reply-To: thong@post.harvard.edu X-Mailer: ELM [version 2.5 PL2] MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit hal@finney.org wrote: > Most people, when they first hear about MN, understand right away the > idea of paying mojo to download files. However they assume that this > means that they will get paid to supply popular files for download. > This is seen as a way to discourage "leeching" and to encourage people > to make material available, which has been a problem on networks like > Gnutella. > > When they discover that they have to pay to put material out into > the network for downloading by others, they become discouraged about > the prospects for MN to succeed in today's crowded P2P environment. The problem seems to me to be that (as far as I know) you must pay mojo immediately if you publish something, but if you donate disk space, you do not receive mojo until later, when someone actually shows up to use the space. This leads to a somewhat nonsensical situation where someone can donate 100 Mb of space to the system, attempt to publish a 1 Kb file, and end up with negative mojo in the short run. Surely you ought to be able to publish unlimited quantities without incurring a net cost, as long as you provide an equivalent amount of storage space. I think that's a condition that most users intuitively accept. theo From hal@finney.org Wed Feb 28 18:05:48 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 18:05:43 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id PAA17463 for bluesky@franklin.oit.unc.edu; Wed, 28 Feb 2001 15:03:01 -0800 Date: Wed, 28 Feb 2001 15:03:01 -0800 From: hal@finney.org Message-Id: <200102282303.PAA17463@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Naming via SDSI I've been studying OceanStore [1], which uses a SDSI [2] like system for "friendly names". This is similar to what Ted Anderson proposed in an earlier message. Here is how one of the OceanStore papers describes it: Certain OceanStore objects act as directories, mapping human-readable names to GUIDs. To allow arbitrary directory hierarchies to be built, we allow directories to contain pointers to other directories. A user of the OceanStore can choose several directories as "roots" and secure thsoe directories through external methods, such as a public key authority. Note, however, that such root directoreis are only roots with respect to the clients that use them; the system as a whole has no one root. This scheme does not solve the problem of generating a secure GUID mapping, but rather reduces it to a problem of secure key lookup. We addresss this problem using the locally linked name spaces from the SDSI framework. The idea, then, is to have a directory hierarchy similar to what we are familiar with from file systems. As with Unix, files can be linked into more than one directory (and unlike with Unix, directories can be, too). Ted gave examples like /com/us/microsoft or /entertainment/music/jazz. With SDSI, each component of such names is associated with a public key. The public key signs bindings of the names below it in the hierarchy to their own keys. In the example above, the key associated with "/us" signs a certificate binding "com" to its own key; and that key signs a certificate binding "microsoft" to the Microsoft key. The certified information would also include hints about where in the network to find the directories associated with each name, and possibly other data. So to use it, if someone gave you the name /com/us/microsoft, and you knew (or could find) the top-level address and key of /com, you'd look up there and ask for the cert for "us". You would check the sig on this cert (signed by the "com" key) and then go to the address for "us" and repeat the process to get the cert for "microsoft". This is also similar to how the Secure DNS system will work. microsoft.us.com would be looked up by starting with the key and DNS server for ".com", and finding the sig on the record for "us", which includes the key for "us" as well as its IP address. Then you'd go to "us" and get the same information for "microsoft", etc. This scheme has a lot of nice properties, including security and flexibility. Each person can have his own chunk of name space where he can manage any filenames he wants below it. And multiple hierarchies can exist at once, overlayed on the same data. Both directories and files can be mapped into multiple other directories. However I think it has significant drawbacks as well. One is the special handling necessary for top-level directories (I will call them TLDs as they play essentially the same role in DNS). For a global name like those above to be meaningful, the user of the name must have the key and address of the TLD. If you see a name like "/bluesky/mailinglist/archives" and you've never heard of "/bluesky", it will be a problem for you to find the data. Some other mechanism may have to be introduced to index TLDs. Furthermore, this scheme seems likely to inherit some of the same problems of TLDs in the DNS world. In practice it has not proven possible to easily expand the set of widely recognized TLDs. There are a number of organizations providing name service which let anyone add their own TLDs, but few people use them because few people use them. In other words, if you use an oddball TLD that nobody else knows, your address won't work. So there is social and economic pressure to converge on a single set of TLDs that everyone recognizes. Then, in this situation, the political problems surrounding DNS TLDs may become present as well. Getting a new TLD established may require convincing some kind of standards body to approve it so that everyone will start supporting it. The controversies over the actions of ICANN show what a bad situation this is. There is also the problem of forcing higher level directories (including but not limited to TLDs) to remove entries which lead down to controversial data. We see this to some extent with the IP hierarchy, where censors go after ISPs or upstream providers to try to silence users who refuse to shut up. Similar problems arise with the "blackhole lists" used to punish spammers; these are used for all network connectivity, not just email, and sometimes wipe out innocent bystanders. The peacefire.org site is unavailable to many internet users because it is hosted by a provider who has also had some spammers. If your data is widely known by a certain address, and a component of that gets knocked out, the data may become effectively unavailable. Even if an alternate hierarchy points to it, if that hierarchy is not widely supported it won't help, and at best it will take time to circulate information about the new address. Controversial sites today are often forced to hop all over the net, able to stay at any one address for only a few weeks until someone complains. This makes them almost impossible to find. Another problem with this scheme is that I don't see that it extends very well to metadata. Binding a directory name to a key is a mechanical process and doesn't imply any endorsement. Binding metadata to a file is another matter and probably requires human interaction, at least if the binding is going to mean anything. You might also see some of the other problems with DNS, like an ultra-wide hierarchy to allow shorter names. There are something like 10 million .com addresses. You need a whole infrastructure just to service that one TLD. Such a system is not an attractive prospect for a SDSI name space. Hal [1] http://oceanstore.cs.berkeley.edu [2] http://theory.lcs.mit.edu/~cis/sdsi.html From pete@retep.yi.org Wed Feb 28 18:44:51 2001 Return-Path: Received: from femail3.rdc1.on.home.com ([24.2.9.90]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 18:44:47 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail3.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010228234358.FONY16595.femail3.rdc1.on.home.com@gw.localdomain> for ; Wed, 28 Feb 2001 15:43:58 -0800 Received: from pete by gw.localdomain with local (Exim 3.16 #1 (Debian)) id 14YGGf-0004NQ-00 for ; Wed, 28 Feb 2001 18:44:25 -0500 Date: Wed, 28 Feb 2001 18:44:25 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010228184425.A16463@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="gBBFr7Ir9EOA20Yy" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from blanu@uts.cc.utexas.edu on Tue, Feb 27, 2001 at 10:43:04PM -0600 Sender: Peter Todd --gBBFr7Ir9EOA20Yy Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Feb 27, 2001 at 10:43:04PM -0600, Brandon wrote: >=20 > > Popularity is only one reason to store data. Some data may not be popul= ar > > but may have high value for a small group of people or have long-term > > archival value. I guess Freenet focuses on popularity because it's hard= er > > to manipulate than subjective value, but the downside is that it can't > > handle the latter kind of data. A market-based system can automatically > > and optimally balance between the two. >=20 > Popularity is local, not global, in Freenet. So if your small group of > people with shared values have nodes which shared proximity then they will > serve as an island in which that information will survive. It will Wouldn't you get clusters of nodes with common interests but with routing fscked up in them because the routing is based on key closeness, not "human" closeness? This sounds ok to so long as the data used by the human getting information from the node doesn't drown out the "correct" data (based on what the network thinks the node has) for the node. Otherwise we run into problems... Decreasing the "value" of locally requested data, IE drop it in favor of non-locally requested data if the choice needs to be made, could probably help this but I for one would hack my node to not do this to get better caching of what I want. I wonder if the Freenet sims done with the nodes requesting a specific set of data each (with overlaping groups) to more closely replecate what real users do? If so, meaning that such effects aren't all that problematic, then I stand corrected. --=20 retep@penguinpowered.com http://retep.tripod.com=20 --gBBFr7Ir9EOA20Yy Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6nY1YLdZCJKx+dn0RAjJvAJ0cOzdU+U+nNxF6Koft1SQB09iBSgCdGbJ8 ap92mfeqk9ZL2umiEep9STk= =tx1j -----END PGP SIGNATURE----- --gBBFr7Ir9EOA20Yy-- From wesley@felter.org Wed Feb 28 19:48:23 2001 Return-Path: Received: from valentine.felter.org ([64.39.30.19]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 19:48:19 -0500 Received: from localhost (wesley@localhost) by valentine.felter.org (8.9.3/8.9.3) with ESMTP id SAA03609 for ; Wed, 28 Feb 2001 18:49:52 -0600 Date: Wed, 28 Feb 2001 18:49:51 -0600 (CST) From: Wesley Felter To: bluesky@franklin.oit.unc.edu Subject: Paying to publish Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII I just subscribed and read the archives, so instead of trying to reply to a previous message I'll just start a new thread. I've been using Mojo Nation for a few months, and I get the impression that I make a profit by publishing files. (I haven't analyzed what data I'm making mojo from, but after publishing a file I see a surge in customers and profit.) This is possible because 1/8 of the blocks that I publish are actually published to myself, and if I sell 2 or 3 copies of each one then I make a net profit. Publishing a block is currently cheaper than downloading it, perhaps because most brokers are not full (or maybe because that's simply the default setting). The 2X data expansion caused by file splitting is still annoying, though. Wesley Felter - wesley@felter.org - http://felter.org/wesley/ From md98-osa@nada.kth.se Wed Feb 28 19:51:22 2001 Return-Path: Received: from mail1.telia.com ([195.67.239.185]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 19:51:17 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14YHLV-0000ks-00 for bluesky@franklin.oit.unc.edu; Thu, 01 Mar 2001 01:53:29 +0100 Date: Thu, 1 Mar 2001 01:53:29 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010301015329.C2677@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Wed, Feb 28, 2001 at 03:03:01PM -0800 On Wed, Feb 28, 2001 at 03:03:01PM -0800, hal@finney.org wrote: <> > With SDSI, each component of such names is associated with a public key. > The public key signs bindings of the names below it in the hierarchy > to their own keys. In the example above, the key associated with "/us" > signs a certificate binding "com" to its own key; and that key signs a > certificate binding "microsoft" to the Microsoft key. I would argue that this is a client application, possible in any network that supports signed subspaces. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From blanu@uts.cc.utexas.edu Wed Feb 28 20:09:38 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 20:09:33 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id TAA03197 for ; Wed, 28 Feb 2001 19:12:30 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Wed, 28 Feb 2001 19:12:30 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > With SDSI, each component of such names is associated with a public key. > The public key signs bindings of the names below it in the hierarchy > to their own keys. In the example above, the key associated with "/us" > signs a certificate binding "com" to its own key; and that key signs a > certificate binding "microsoft" to the Microsoft key. Freenet contains a similar mechanism called Mapped Subspace Keys (MSK). An MSK contains a key to the mapfile and a name to map. If a mapfile's key is a signed key (SVK or SSK) then all of the mappings are of course signed with the mapfile's key's public key (did that make sense?). The main difference I see between Freenet and OceanStore in this respect is that Freenet MSKs are not broken up into components so /com/us/microsoft would be treated as an atomic name. I can see the benefit of the other system. This wasn't chosen in Freenet due to the overhead of fetching a lot of files to resolve a path and because people want to use Freenet as if it's the web and therefore complain a lot of latency. > One is the special handling necessary for top-level directories (I > will call them TLDs as they play essentially the same role in DNS). > For a global name like those above to be meaningful, the user of the > name must have the key and address of the TLD. If you see a name like > "/bluesky/mailinglist/archives" and you've never heard of "/bluesky", > it will be a problem for you to find the data. Some other mechanism > may have to be introduced to index TLDs. This is the reason that in Freenet MSKs we made the mapfile key an intrinsic part of the key. For top level mappings you must either have 1) a central authority, 2) no authority and therefore only second-visit trust, as per SSH (in Freenet this is done with a KSK redirect to the MSK), 3) you choose the authorities you trust, as in PGP, which really won't work for DNS. > Then, in this situation, the political problems surrounding DNS TLDs > may become present as well. Getting a new TLD established may require > convincing some kind of standards body to approve it so that everyone > will start supporting it. The controversies over the actions of ICANN > show what a bad situation this is. In a P2P situation ICANN can be replaced or splintered into multiple competing systems. A truly global system requires a single trust authority, however, or else there will be sites that you can't reach. From blanu@uts.cc.utexas.edu Wed Feb 28 20:13:34 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 20:13:31 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id TAA12984 for ; Wed, 28 Feb 2001 19:16:28 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Wed, 28 Feb 2001 19:16:27 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > > Popularity is local, not global, in Freenet. So if your small group of > > people with shared values have nodes which shared proximity then they will > > serve as an island in which that information will survive. It will > > Wouldn't you get clusters of nodes with common interests but with > routing fscked up in them because the routing is based on key > closeness, not "human" closeness? Routing is based on both key closeness and closeness of the nodes on the graph. Obviously you can only send messages to nodes one hop away. If all of these nodes have users with the same interests, you will statistically get the files you want in one hop more often than nodes which exist in an area of the network where no one shares their interests. However, I should make it clear that in the future Freenet network there will be no way to create group of nodes which share proximity and interests. Such things will be determined randomly. From pete@retep.yi.org Wed Feb 28 20:48:13 2001 Return-Path: Received: from femail2.rdc1.on.home.com ([24.2.9.89]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 20:48:09 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail2.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010301014757.XDPJ606.femail2.rdc1.on.home.com@gw.localdomain> for ; Wed, 28 Feb 2001 17:47:57 -0800 Received: from petes.localdomain ([192.168.1.1] ident=mail) by gw.localdomain with esmtp (Exim 3.16 #1 (Debian)) id 14YICI-0004fc-00 for ; Wed, 28 Feb 2001 20:48:02 -0500 Received: from pete by petes.localdomain with local (Exim 3.12 #1 (Debian)) id 14YICE-0000WK-00 for ; Wed, 28 Feb 2001 20:47:58 -0500 Date: Wed, 28 Feb 2001 20:47:58 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010228204758.A1966@petes.localdomain> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="liOOAslEiF7prFVr" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from blanu@uts.cc.utexas.edu on Wed, Feb 28, 2001 at 07:16:27PM -0600 Sender: Peter Todd --liOOAslEiF7prFVr Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Feb 28, 2001 at 07:16:27PM -0600, Brandon wrote: > Routing is based on both key closeness and closeness of the nodes on the > graph. Obviously you can only send messages to nodes one hop away. If all > of these nodes have users with the same interests, you will statistically > get the files you want in one hop more often than nodes which exist in an > area of the network where no one shares their interests. > > However, I should make it clear that in the future Freenet network there > will be no way to create group of nodes which share proximity and > interests. Such things will be determined randomly. How? The current system creates such clusters because of the DataSource address right? You request data you want, nodes that are run by people with your interests are more likely to have the data hence biasing your nodes references, which are derived from the DataSource addresses, to their nodes. It's probably a relatively subtle effect but it is there. How on earth would you wreck this without wrecking routing completely? --=20 retep@penguinpowered.com http://retep.tripod.com=20 --liOOAslEiF7prFVr Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6napOLdZCJKx+dn0RAubjAJ9MEeYMXQ03Pncvv4HeFK62Vhit3wCcC4Wq My0uWTcunCgb5t7YkccW3A4= =IKFl -----END PGP SIGNATURE----- --liOOAslEiF7prFVr-- From hal@finney.org Wed Feb 28 21:00:39 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 21:00:35 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id RAA18105 for bluesky@franklin.oit.unc.edu; Wed, 28 Feb 2001 17:57:54 -0800 Date: Wed, 28 Feb 2001 17:57:54 -0800 From: hal@finney.org Message-Id: <200103010157.RAA18105@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI Oskar writes, regarding SDSI-style naming: > I would argue that this is a client application, possible in any network > that supports signed subspaces. This is in the context of Wei Dai's proposed three-services model [1], 1. name lookup service: maping user-friendly names/URLs to content-hash based ids 2. data location service: maping content-hash based ids to storage servers (network addresses and data transport protocols) that can deliver the data 3. data transport service: actually obtaining content from a storage server given its address and a content-hash based id I wanted to raise some issues with how the first service might work. Whether it is a client or a network service is a good question, but independent of that there are still questions about how well the SDSI model works in general. Hal [1] http://www.mail-archive.com/bluesky%40franklin.oit.unc.edu/msg00012.html From md98-osa@nada.kth.se Wed Feb 28 21:12:54 2001 Return-Path: Received: from mail1.telia.com ([195.67.239.211]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 21:12:47 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14YIcN-0000qQ-00 for bluesky@franklin.oit.unc.edu; Thu, 01 Mar 2001 03:14:59 +0100 Date: Thu, 1 Mar 2001 03:14:59 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments Message-ID: <20010301031459.A3084@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from retep@penguinpowered.com on Wed, Feb 28, 2001 at 08:47:58PM -0500 On Wed, Feb 28, 2001 at 08:47:58PM -0500, Peter Todd wrote: > On Wed, Feb 28, 2001 at 07:16:27PM -0600, Brandon wrote: <> > How? The current system creates such clusters because of the > DataSource address right? You request data you want, nodes that are > run by people with your interests are more likely to have the data > hence biasing your nodes references, which are derived from the > DataSource addresses, to their nodes. It's probably a relatively > subtle effect but it is there. In order to reply to the request, nodes not only have to have the data, they have to get the request sent to them in the first place. The idea behind Freenet is not that the requests go where the data is, but that the data goes where the requests are. > > How on earth would you wreck this without wrecking routing completely? > > -- > retep@penguinpowered.com http://retep.tripod.com Content-Description: footer > --- > You are currently subscribed to bluesky as: md98-osa@nada.kth.se > For list information visit http://www.transarc.ibm.com/~ota/bluesky/ -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From blanu@uts.cc.utexas.edu Wed Feb 28 23:14:37 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 28 Feb 2001 23:14:24 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id WAA09818 for ; Wed, 28 Feb 2001 22:17:20 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Wed, 28 Feb 2001 22:17:20 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Hi, and some comments In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > How? The current system creates such clusters because of the > DataSource address right? I don't think that the people on the bluesky list really want to hear so much about Freenet routing. I was just mentioning Freenet in reference to more general concepts. If you want to discuss Freenet routing in depth, we should probably take it to the Freenet chat list. From hal@finney.org Thu Mar 01 14:27:24 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 01 Mar 2001 14:27:08 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA20764 for bluesky@franklin.oit.unc.edu; Thu, 1 Mar 2001 11:24:24 -0800 Date: Thu, 1 Mar 2001 11:24:24 -0800 From: hal@finney.org Message-Id: <200103011924.LAA20764@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: the three-services model Wei Dai's original article on the Three Services Model has appeared on the openp2p.com news ticker (look at the lower right part of that page, or the news page at http://www.openp2p.com/p2p/news.csp). The link ponints directly into the Bluesky archives. The OpenP2P site does not provide a discussion forum on its news page, but this could lead to wider distribution of information about this list. Hal From mccoy@io.com Mon Mar 05 20:43:34 2001 Return-Path: Received: from oppenheimer.mad-scientist.com ([216.15.44.90]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 05 Mar 2001 20:43:11 -0500 Received: (qmail 8236 invoked from network); 6 Mar 2001 01:43:03 -0000 Received: from groves.mad-scientist.com (192.168.11.254) by oppenheimer.mad-scientist.com with SMTP; 6 Mar 2001 01:43:03 -0000 Received: (from mail@localhost) by groves.mad-scientist.com (8.10.1/8.9.3) id f261h3H11458 for ; Mon, 5 Mar 2001 17:43:03 -0800 (PST) Received: from UNKNOWN(216.15.44.90), claiming to be "gadget.io.com" via SMTP by groves, id smtpdT23720; Tue Mar 6 01:42:58 2001 Message-Id: <4.3.2.7.2.20010302142754.00d11290@pop.io.com> X-Sender: mccoy@pop.io.com X-Mailer: QUALCOMM Windows Eudora Version 4.3.2 Date: Mon, 05 Mar 2001 17:41:24 -0800 To: "Global-Scale Distributed Storage Systems" From: Jim McCoy Subject: Re: market-based data location (was Scalability) In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed At 03:02 AM 2/28/01 +0100, Oskar Sandberg wrote: >[...] >I approached the Mojonation people regarding my concern that the whole >content tracking system was not really decentralized at all at the Peer to >Peer conference. Actually our opinion is that our content tracking system is _too_ decentralized. It would have served us better to start off with a few centralized content trackers (a la Napster) that were specialists in a particular content type rather than the Gnutella-like indexing structure which we currently employ. A few handicappers exist within our current code to do keyword-hash based lookups (woe to the poor slob who gets stuck with "britanny" or "spears" :) but these have not really been tested very much. >It turned out however, that Mojonation's basic architecture isn't scalable >anyways. The content-trackers in MN do not map metadata onto a physical >address, they map metadata onto UIDs (the SHA1 hash for each part) of the >data. Apparently, to then find the data, MN simply expects each node to >keep a list of the IDs of every other node in the network [...] This is incorrect. Each node keeps a set of phonebook entries for other brokers that it has contacted in the past (not the entire list, just the nodes you have communicated with). These entries contain information such as broker ID (to link to reputation information about that broker), contact information and connectin strategies (e.g. to contact Broker X send the message to relay server Y), and specific service info such as the block mask for a storage service agent. It is these masks which are used to determine who you will send a request to when looking for a blob of data. If you can't find the blob you are looking for at any of the brokers you know who cover that range of the SHA1 address space then the Broker goes back to the centralized metatracker and asks it for a few more phonebook entries which cover the range of the address space it is interested in and caches these entries. This scales sufficiently well for our current prototype and when this starts to show scaling problems we will drop in the gossip-based mechanism to limit reliance on metatrackers for new block server phonebook entries. In our gossip-based lookup system your Broker would send a phonebook lookup request to block servers that a Broker knows about which were closest to the desired range. These block servers would either answer the request as best they could with entries out of thier cache or direct the originator of the request to a few other block servers which were even closer to the correct mask range in the hope that they could provide useful phonebook info. Each Broker will hold on to phonebook entries that overlap or are near its mask range longer than it will for other ranges of the block address space. Each block server within a Broker will become a specialist in the metainfo about other blocks servers whose mask is a near-match, allowing us to replicate data blocks within the network easier and assisting other Brokers by being able to provide useful phonebook info. Next time perhaps you could try to make sure that you actually understand how another system works before you start throwing stones... jim mccoy AZI/Mojo Nation From md98-osa@nada.kth.se Tue Mar 06 06:23:36 2001 Return-Path: Received: from mail1.telia.com ([62.20.149.205]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 06:23:28 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14aFbC-0000Fd-00 for bluesky@franklin.oit.unc.edu; Tue, 06 Mar 2001 12:25:50 +0100 Date: Tue, 6 Mar 2001 12:25:50 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: market-based data location (was Scalability) Message-ID: <20010306122550.C623@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from mccoy@io.com on Mon, Mar 05, 2001 at 05:41:24PM -0800 On Mon, Mar 05, 2001 at 05:41:24PM -0800, Jim McCoy wrote: > At 03:02 AM 2/28/01 +0100, Oskar Sandberg wrote: > >[...] > >I approached the Mojonation people regarding my concern that the whole > >content tracking system was not really decentralized at all at the Peer to > >Peer conference. > > Actually our opinion is that our content tracking system is _too_ > decentralized. It would have served us better to start off with a few > centralized content trackers (a la Napster) that were specialists in a > particular content type rather than the Gnutella-like indexing structure > which we currently employ. A few handicappers exist within our current > code to do keyword-hash based lookups (woe to the poor slob who gets stuck > with "britanny" or "spears" :) but these have not really been tested very much. I guess that with the emphasis of MN's technology research being on the monetary system you can justify doing a rough job of searching - but I think you can understand why those of us who emphasize on data sharing architectures are somewhat taken aback by that. Especially in light of MN often being presented as a decentralized data sharing network. > >It turned out however, that Mojonation's basic architecture isn't scalable > >anyways. The content-trackers in MN do not map metadata onto a physical > >address, they map metadata onto UIDs (the SHA1 hash for each part) of the > >data. Apparently, to then find the data, MN simply expects each node to > >keep a list of the IDs of every other node in the network [...] > > This is incorrect. Each node keeps a set of phonebook entries for other > brokers that it has contacted in the past (not the entire list, just the > nodes you have communicated with). These entries contain information such > as broker ID (to link to reputation information about that broker), contact > information and connectin strategies (e.g. to contact Broker X send the > message to relay server Y), and specific service info such as the block > mask for a storage service agent. It is these masks which are used to > determine who you will send a request to when looking for a blob of data. > > If you can't find the blob you are looking for at any of the brokers you > know who cover that range of the SHA1 address space then the Broker goes > back to the centralized metatracker and asks it for a few more phonebook > entries which cover the range of the address space it is interested in and > caches these entries. This scales sufficiently well for our current > prototype and when this starts to show scaling problems we will drop in the > gossip-based mechanism to limit reliance on metatrackers for new block > server phonebook entries. Obviously, I don't think I need to comment on my opinion of this model. Are you guys not concerned about waking up one morning with a friendly letter from the RIAA in your mailbox (something that has already happened to several Opennap admins)? > In our gossip-based lookup system your Broker would send a phonebook lookup > request to block servers that a Broker knows about which were closest to > the desired range. These block servers would either answer the request as > best they could with entries out of thier cache or direct the originator of > the request to a few other block servers which were even closer to the > correct mask range in the hope that they could provide useful phonebook > info. Each Broker will hold on to phonebook entries that overlap or are > near its mask range longer than it will for other ranges of the block > address space. Each block server within a Broker will become a specialist > in the metainfo about other blocks servers whose mask is a near-match, > allowing us to replicate data blocks within the network easier and > assisting other Brokers by being able to provide useful phonebook info. This is closer to the sort of the self-sorting we are using for Freenet, although we are interested only in comparing the data key (aka GUID or whatever) values, since nodes have no static range to cover (let alone one they can choose themselves, since we consider this opportunity for targeted attacks on parts of the keyspace unacceptable). I can warn you however, that making this sort of system work is not easy - and even when it does you have given up deterministically finding the results. In light of the fact that you are using the "rigid" model of matching data id's and with node id's anyways, I would very recommend you look at the Plaxton model for routing in this type of system. It's very fast for searching (you can keep it 4 or 8 times under logarithmic for the worst case) and also has some interesting characteristics for finding the best located data (though you needn't employ those). > Next time perhaps you could try to make sure that you actually understand > how another system works before you start throwing stones... Not a chance. > > jim mccoy > AZI/Mojo Nation -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From ota@transarc.com Tue Mar 06 06:53:29 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 06:53:23 -0500 Received: from transarc.com (transarc.com [9.38.192.8]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id GAA15132 for ; Tue, 6 Mar 2001 06:47:45 -0500 (EST) Received: by transarc.com (8.9.0.Beta3/3.15) id for bluesky@franklin.oit.unc.edu; Tue, 6 Mar 2001 06:53:16 -0500 (EST) Received: via switchmail; Tue, 6 Mar 2001 06:53:16 -0500 (EST) Received: from smithfield via qmail ID ; Tue, 6 Mar 2001 06:52:21 -0500 (EST) Date: Tue, 6 Mar 2001 06:52:17 -0500 (EST) Message-Id: To: bluesky@franklin.oit.unc.edu Subject: Re: market-based data location (was Scalability) In-Reply-To: References: From: Ted Anderson On 27 Feb 2001 00:12:55 PST wrote: > But I was struck by the statement in the introduction to one of their > papers that their goal is to support a network with 10^10 users (10 > billion users, i.e. the total population of Earth), each storing 10000 > files, for a total of 100 trillion files. That's a lot of files! > Scaling to that level will be a real challenge. This figure struck me as well, because it agreed with my thinking about the appropriate scale of the decentralized file system I envision. This was crystallized for me when I tried, unsuccessfully, to convince myself that a 128-bit hash value was sufficiently large for object CHKs. When I considered how many nodes might participate in the system and how many objects each might create over a period of years, I could not do it. I assumed 10^10 nodes (not users, but actively participating computers) and over its lifetime it seemed plausible that a node might create 10^10 files (about 33/sec for 10 years). This comes to 10^20 (2^66.5) objects and, since the birthday paradox tells us that collisions will be common when we reach the square root of the hash size, 128 bits seemed just too small for comfort. Using the 160 bit SHA-1 output provides a nice margin of safety. In a relatively benign environment, I could imagine a system like the OceanStore data location scheme working at this scale. I have argued in favor of a similar idea based on boolean hypercubes for several years. However, I now think the Internet is hostile enough that a more flexible, adaptable approach based on market incentives will be necessary. At the beginning of this thread, Wei Dai, very briefly, described such a system. The economic model is the only one we know of that can organize loosely-coupled human systems on this scale. If we generalize the interpretation of "economic" to include cooperative, mutually beneficial interactions of all kinds this organizational capability applies to complex systems of essentially any type. Robert Wright, expounds on this idea in his book titled "Nonzero: The logic of Human Destiny". He argues that increasing complexity of all living systems are driven by nonzero-sum interactions on many levels. The specific shortcomings of MojoNation-style economics, and with micro-payments in general, are worth understanding. The conclusion I think we must draw, however, is not that we should avoid market-based incentives, but that we need to find a way to incorporate these mechanisms in a way that will enable mutually beneficial cooperation between agents while suppressing parasitic interactions. Ted Anderson From hal@finney.org Tue Mar 06 12:08:46 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 12:08:30 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id JAA13116 for bluesky@franklin.oit.unc.edu; Tue, 6 Mar 2001 09:05:31 -0800 Date: Tue, 6 Mar 2001 09:05:31 -0800 From: hal@finney.org Message-Id: <200103061705.JAA13116@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI An article on Wired Online this morning describes a case of multiple name services competing for business. One of the issues is that it will not work well if there are two independent name server systems that service the same TLD (.xxx in this case). I think the problems are, first, that it would be inefficient to have to query two name servers to find out where sexygirls.xxx is; and second, that the two groups might register the same name, making addresses ambiguous. Both of these would be resolved by assigning each TLD to only one name service operator. How would that work in a P2P model? If we did have competing name lookup services (using Wei's term from the three service model), would there have to be some global mechanism to make sure that no two services tried to handle the same addresses? Or would we accept that address to name mapping was inherently ambiguous? Or, perhaps, would TLDs have to have unfriendly unforgeable names? Mark Miller sent me a pointer to an idea of his that he had proposed in another context, http://www.erights.org/elib/capability/pnml.html. The idea of his "Pet Name Markup Language" (pnml) is that you pass around long, unambiguous names (such as hashes), but let the display software substitute short, user-assigned, local nicknames, somewhat similar to what is done in SDSI. Maybe this could eliminate the need for name lookup services, at least in some cases (including in particular the TLD issue discussed above). Hal === http://www.wired.com/news/business/0,1367,42217,00.html: New.net announced Monday that it was offering 20 new top-level domains (TLDs) including dot-xxx, which could be intriguing for adult sites. But Domain Name Systems has been selling dot-xxx domains for several months. The competing efforts that go outside the Internet's accepted domain name system could become a complex legal dispute over dispensing adult domains. New.net began selling domains using TLDs that are not approved by the Internet Corporation for Assigned Names and Numbers (ICANN), the international agency charged with overseeing TLDs. The company began distributing software that enables Web browsers to access the new TLDs. Domain Name Systems has been offering dot-xxx and dot-sex registrations since ICANN rejected those two proposed TLDs last November. ... Stacey King, an intellectual property lawyer at Howrey Simon Arnold & White, said disputes between two companies trying to register the same domain name would have to be settled via trademark, if it's possible. "Things will have to go back to traditional trademark principles. Do they have a registered trademark or common law rights to a name? Have they used it in commerce? They may have greater rights than someone who doesn't have a federal registration," she said. If there is no trademark or history of doing business under a name, then things could get nasty, since there are no laws governing ownership of top-level domains. "It may come down to first-come, first-served unless these two companies want to fight it out in the courts who has a right to the top-level domain," King said. From markm@caplet.com Tue Mar 06 12:21:44 2001 Return-Path: Received: from proxy4.ba.best.com ([206.184.139.15]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 12:21:26 -0500 Received: from caplet.caplet.com (sdn-ar-006carcorP169.dialsprint.net [63.178.230.209]) by proxy4.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id JAA10867; Tue, 6 Mar 2001 09:19:09 -0800 (PST) Message-Id: <5.0.2.1.2.20010306091536.00a88290@shell9.ba.best.com> X-Sender: erights@shell9.ba.best.com X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 06 Mar 2001 09:17:14 -0800 To: "Global-Scale Distributed Storage Systems" From: "Mark S. Miller" Subject: Re: Naming via SDSI Cc: "Global-Scale Distributed Storage Systems" In-Reply-To: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 09:05 AM Tuesday 3/6/01, hal@finney.org wrote: >Mark Miller sent me a pointer to an idea of his that he had proposed >in another context, http://www.erights.org/elib/capability/pnml.html. >The idea of his "Pet Name Markup Language" (pnml) is that you pass around >long, unambiguous names (such as hashes), but let the display software >substitute short, user-assigned, local nicknames, somewhat similar to >what is done in SDSI. Maybe this could eliminate the need for name >lookup services, at least in some cases (including in particular the >TLD issue discussed above). Thanks! I'm puzzled by this reference to SDSI. I had thought that SDSI had been fully absorbed into SPKI, and AFAIK SPKI has no such mapping from keys back to names. ?? Cheers, --MarkM From hal@finney.org Tue Mar 06 12:42:49 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 12:42:38 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id JAA13361 for bluesky@franklin.oit.unc.edu; Tue, 6 Mar 2001 09:39:36 -0800 Date: Tue, 6 Mar 2001 09:39:36 -0800 From: hal@finney.org Message-Id: <200103061739.JAA13361@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI I wrote earlier: > Mark Miller sent me a pointer to an idea of his that he had proposed > in another context, http://www.erights.org/elib/capability/pnml.html. > The idea of his "Pet Name Markup Language" (pnml) is that you pass around > long, unambiguous names (such as hashes), but let the display software > substitute short, user-assigned, local nicknames, somewhat similar to > what is done in SDSI. I want to clarify that "what is done in SDSI" is maintaining a database associating local nicknames with keys. SDSI does not map back and forth between nicknames and keys the way Mark proposes in the PNML. That is a new and interesting idea which could be another approach to the name mapping problem. Hal From dmarti@zgp.org Tue Mar 06 14:09:30 2001 Return-Path: Received: from capsicum.zgp.org ([198.144.202.250]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 14:09:22 -0500 Received: by capsicum.zgp.org (Postfix, from userid 1000) id 7417E3FC21; Tue, 6 Mar 2001 11:09:14 -0800 (PST) Date: Tue, 6 Mar 2001 11:09:14 -0800 From: Don Marti To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010306110914.F3154@zgp.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i In-Reply-To: ; from hal@finney.org on Tue, Mar 06, 2001 at 09:05:31AM -0800 On Tue, Mar 06, 2001 at 09:05:31AM -0800, hal@finney.org wrote: > How would that work in a P2P model? If we did have competing name lookup > services (using Wei's term from the three service model), would there > have to be some global mechanism to make sure that no two services tried > to handle the same addresses? Or would we accept that address to name > mapping was inherently ambiguous? Or, perhaps, would TLDs have to have > unfriendly unforgeable names? All this bogus DNS stuff will get settled at the business level, not the crypto level. There are already sites that rate registrars. The services would just have to add a column to their ratings chart with an estimate of what percentage of the hosts on the net can resolve each registrar's domains. (Data for this could be collected by putting small images, srced from all the new domains, on popular pages.) New registrars will presumably offer registrations at no charge to build up a user base, and pay webmasters to make new-domain-only content to encourage ISPs to recognize their TLDs. (For example, if you run pornpornporn.com, a new registrar offering .xxx will _give_ you pornpornporn.xxx and pay you to put free porn there that's not on your original site.) More popular registrars will get more registrations with more content, thereby scooping up more ISPs -- positive reinforcement for the more popular one that will make the less popular one get fewer registrations, less content, and less demand. Competition won't be an issue. -- Don Marti "I've never sent or received a GIF in my life." dmarti@zgp.org -- Bruce Schneier, Secrets and Lies, p. 246. http://zgp.org/~dmarti/ (Free the Web: http://burnallgifs.org/) From hal@finney.org Tue Mar 06 14:46:08 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 14:46:00 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA13981 for bluesky@franklin.oit.unc.edu; Tue, 6 Mar 2001 11:42:58 -0800 Date: Tue, 6 Mar 2001 11:42:58 -0800 From: hal@finney.org Message-Id: <200103061942.LAA13981@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI Don Marti, , writes: > More popular registrars will get more registrations with more content, > thereby scooping up more ISPs -- positive reinforcement for the more > popular one that will make the less popular one get fewer registrations, > less content, and less demand. Competition won't be an issue. You are suggesting that the end result of a competitive process will be that each new TLD is assigned to just one registrar (name lookup service). Each one will have a de facto monopoly on that particular name. It's possible that the results will approximate this state, but I don't think it will be as clean as you suggest. New domains will spring up all the time and there will be a battle over who gets to serve them. Well established registrars may make "raids" on other registrars' domains to try to establish themselves as alternatives for those TLDs. The same thing can be accomplished by creating new domains: is it .xxx or .sex? .vid or .tv? .snd or .mus? Registrars will spend money promoting new alternative TLDs in the hopes that they will become popular and steal business from other TLDs. There may also be truces between registrars who agree to share a TLD and cooperate to make sure they don't assign competing names. Then these truces will be broken sometimes. It's going to be a dynamic process. That's how competition works. It seems likely that a substantial number of TLDs will be in flux at any given time. This will lead to ambiguity, uncertainty and costs in doing name lookups. I don't think this is an ideal solution. Hal From dmarti@zgp.org Tue Mar 06 20:47:06 2001 Return-Path: Received: from capsicum.zgp.org ([198.144.202.250]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 20:47:01 -0500 Received: by capsicum.zgp.org (Postfix, from userid 1000) id D97A33FC0A; Tue, 6 Mar 2001 17:46:52 -0800 (PST) Date: Tue, 6 Mar 2001 17:46:52 -0800 From: Don Marti To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010306174652.H19251@zgp.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i In-Reply-To: ; from hal@finney.org on Tue, Mar 06, 2001 at 11:42:58AM -0800 On Tue, Mar 06, 2001 at 11:42:58AM -0800, hal@finney.org wrote: > It's going to be a dynamic process. That's how competition works. > It seems likely that a substantial number of TLDs will be in flux at > any given time. This will lead to ambiguity, uncertainty and costs in > doing name lookups. I don't think this is an ideal solution. A two-tier structure of both competitive and official TLDs would be better than an ICANN-only one. People who want to explore the wild and wooly domains can run a simple caching nameserver at home; people who want to stick with ICANN and any rogue TLDs that get well-established can use their ISP's name server. -- Don Marti "I've never sent or received a GIF in my life." dmarti@zgp.org -- Bruce Schneier, Secrets and Lies, p. 246. http://zgp.org/~dmarti/ (Free the Web: http://burnallgifs.org/) From blanu@uts.cc.utexas.edu Tue Mar 06 22:09:44 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:09:32 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id VAA10138 for ; Tue, 6 Mar 2001 21:09:26 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Tue, 6 Mar 2001 21:09:25 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > How would that work in a P2P model? If we did have competing name lookup > services (using Wei's term from the three service model), would there > have to be some global mechanism to make sure that no two services tried > to handle the same addresses? Or would we accept that address to name > mapping was inherently ambiguous? Or, perhaps, would TLDs have to have > unfriendly unforgeable names? Well, I see three ways to handle this: central authority to resolve disputes, choosing sides, or total subjectivity. Pet names is total subjectivity. I think that's a pretty good system. It's the system I'm using to do e-mail over Freenet (PGP fingerprint == key, Freenet "e-mail address" == pet name). My main issue with total subjectivity is that it requires that you pass around fat references (keys) all the time. So I can't tell someone "Check out megatokyo.com." I have to say "check out asjldkq9u4q8412093ajla0912l". It helps if you have a computer-mediated communication environment to do translation for you, but it sucks for voice and written. I know the last time I wrote down my PGP fingerprint I made to copying errors. If it was a different kind of key, such errors woul have made it useless. Having naming paths helps because then, for instance, I can run a name mapping service and if you already have my key then I can tell you "check out brandon/megatokyo" and that path will resolve. It still has the same theoretical problems, but in practice will suck less because you tend to communicate with the same people a lot. From blanu@uts.cc.utexas.edu Tue Mar 06 22:13:08 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:12:59 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id VAA10224 for ; Tue, 6 Mar 2001 21:12:52 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Tue, 6 Mar 2001 21:12:52 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > There are already sites that rate registrars. The services would just > have to add a column to their ratings chart with an estimate of what > percentage of the hosts on the net can resolve each registrar's domains. > (Data for this could be collected by putting small images, srced from > all the new domains, on popular pages.) This is just moving the trust authority to the registrar rating site. Or to a trust oligopoly of registrar rating sites. So I'd say it doesn't actually change the problem at all. You now have to choose you to trust, you just have a new layer of middlemen. I'm not saying this isn't better than ICANN. I think it is. From alk@pobox.com Tue Mar 06 22:19:11 2001 Return-Path: Received: from poboxer.pobox.com ([199.199.248.17]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:18:58 -0500 Received: from spanky.love.edu (spanky [172.30.0.35]) by poboxer.pobox.com (8.9.3/8.9.3) with SMTP id VAA31298 for ; Tue, 6 Mar 2001 21:18:20 -0600 From: Tony Kimball MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15013.43161.904733.426487@spanky.love.edu> Date: Tue, 6 Mar 2001 21:18:49 -0600 (CST) X-Face: \h9Jg:Cuivl4S*UP-)gO.6O=T]]@ncM*tn4zG);)lk#4|lqEx=*talx?.Gk,dMQU2)ptPC17cpBzm(l'M|H8BUF1&]dDCxZ.c~Wy6-j,^V1E(NtX$FpkkdnJixsJHE95JlhO 5\M3jh'YiO7KPCn0~W`Ro44_TB@&JuuqRqgPL'0/{):7rU-%.*@/>q?1&Ed Reply-To: alk@pobox.com To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI References: X-Mailer: VM 6.72 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Quoth Brandon on Tuesday, 6 March: : : ...resolve disputes... I find it difficult to imagine how disputes could last, frankly: Anyone who plans to stick around is going to be synchronizing with the other registries as often as possible. Indeed, I can't see any sane business reason not to run all the registries off of a single meta-registry database. It's only when someone has the heady smell of monopoly in their nostrils that they could seriously consider disincluding any other competent registry. From pete@retep.yi.org Tue Mar 06 22:39:54 2001 Return-Path: Received: from femail1.rdc1.on.home.com ([24.2.9.88]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:39:44 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail1.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010307033816.UTVB14622.femail1.rdc1.on.home.com@gw.localdomain> for ; Tue, 6 Mar 2001 19:38:16 -0800 Received: from petes.localdomain ([192.168.1.1] ident=mail) by gw.localdomain with esmtp (Exim 3.22 #1 (Debian)) id 14aUnV-0000r1-00 for ; Tue, 06 Mar 2001 22:39:33 -0500 Received: from pete by petes.localdomain with local (Exim 3.12 #1 (Debian)) id 14aUlw-0000PJ-00 for ; Tue, 06 Mar 2001 22:37:56 -0500 Date: Tue, 6 Mar 2001 22:37:55 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010306223755.A1557@petes.localdomain> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="J/dobhs11T7y2rNN" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from blanu@uts.cc.utexas.edu on Tue, Mar 06, 2001 at 09:09:25PM -0600 Sender: Peter Todd --J/dobhs11T7y2rNN Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Mar 06, 2001 at 09:09:25PM -0600, Brandon wrote: > My main issue with total subjectivity is that it requires that you pass > around fat references (keys) all the time. So I can't tell someone "Check > out megatokyo.com." I have to say "check out > asjldkq9u4q8412093ajla0912l". It helps if you have a computer-mediated > communication environment to do translation for you, but it sucks for > voice and written. I know the last time I wrote down my PGP fingerprint I > made to copying errors. If it was a different kind of key, such errors ^ We already make enough errors just writing normal text. :) Whats so special about PGP figerprints BTW that allows them to tolerate errors? --=20 retep@penguinpowered.com http://retep.tripod.com=20 --J/dobhs11T7y2rNN Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6pa0TLdZCJKx+dn0RApGHAJ0YJSlcZbH0EwOAuD92TpI7dlraQwCeK8al O42GMn1Y8kNRhmTspt22X30= =Q1tA -----END PGP SIGNATURE----- --J/dobhs11T7y2rNN-- From blanu@uts.cc.utexas.edu Tue Mar 06 22:46:44 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:46:30 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id VAA11257 for ; Tue, 6 Mar 2001 21:46:24 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Tue, 6 Mar 2001 21:46:24 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > We already make enough errors just writing normal text. :) Names like google, slashdot, and freenetproject are resistant to errors in duplication. > Whats so special about PGP figerprints BTW that allows them to > tolerate errors? You check a PGP fingerprint against a key. So if the fingerprint has errors but not many then you can still trust it. From blanu@uts.cc.utexas.edu Tue Mar 06 22:50:19 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 06 Mar 2001 22:50:01 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id VAA11382 for ; Tue, 6 Mar 2001 21:49:54 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Tue, 6 Mar 2001 21:49:54 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > I find it difficult to imagine how disputes could last, frankly: > Anyone who plans to stick around is going to be synchronizing with the > other registries as often as possible. Indeed, I can't see any sane > business reason not to run all the registries off of a single > meta-registry database. Having a meta-registry is just shifting the central authority from ICANN to the people that run the meta-registry, avoiding the question of a situation without a central trust authority. In such a system disputes might occur. From markm@caplet.com Wed Mar 07 01:46:10 2001 Return-Path: Received: from proxy2.ba.best.com ([206.184.139.14]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 01:45:58 -0500 Received: from caplet.caplet.com (sdn-ar-006carcorP169.dialsprint.net [63.178.230.209]) by proxy2.ba.best.com (8.9.3/8.9.2/best.out) with ESMTP id WAA00486; Tue, 6 Mar 2001 22:45:32 -0800 (PST) Message-Id: <5.0.2.1.2.20010306215651.043f3890@shell9.ba.best.com> X-Sender: erights@shell9.ba.best.com X-Mailer: QUALCOMM Windows Eudora Version 5.0.2 Date: Tue, 06 Mar 2001 22:43:46 -0800 To: "Global-Scale Distributed Storage Systems" From: "Mark S. Miller" Subject: Re: Naming via SDSI Cc: "Global-Scale Distributed Storage Systems" In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 07:09 PM Tuesday 3/6/01, Brandon wrote: >My main issue with total subjectivity is that it requires that you pass >around fat references (keys) all the time. So I can't tell someone "Check >out megatokyo.com." I have to say "check out >asjldkq9u4q8412093ajla0912l". It helps if you have a computer-mediated >communication environment to do translation for you, but it sucks for >voice and written. I know the last time I wrote down my PGP fingerprint I >made to copying errors. If it was a different kind of key, such errors >woul have made it useless. > >Having naming paths helps because then, for instance, I can run a name >mapping service and if you already have my key then I can tell you "check >out brandon/megatokyo" and that path will resolve. It still has the same >theoretical problems, but in practice will suck less because you tend to >communicate with the same people a lot. I think this summarizes well both the good and bad news with total subjectivity, as in SPKI/SDSI/Pet-Names. I also think this is as good as it gets. See http://www.cap-lore.com/MathPhys/GRref.html . And I think the bad news isn't this bad, as explained below. The other choices are mostly illusory. "Central authority" actually becomes "choosing sides" since (hopefully) we'll never have unanimity about who should be world dictator. Witness the upcoming war between ICANN and the government of Tonga of the TLD ".to". (Tonga refuses to pay their dues to ICANN, saying, rightly, "Who put you in charge?".) Global name spaces create unsolvable political problems. To attempt to live within a global name space is to choose to subjugate yourself and your customers to unnecessarily centralized authorities. "Choosing sides" is really just a broken way to look at subjectivity + naming paths anyway, as the creators of SDSI correctly explained. For example, in my use of PNML, BrandontldServertocypherpunks starts with the name server I take to serve Brandon's exported names, asks it what TLD server it takes as authoritative, and then decodes cypherpunks.to within that TLD system. This might be a different entity than the one designated by my use of "tldServertocypherpunks". So even the domain name system is best seen through the eyes of subjectivity + naming paths. For direct communication with people you already know, I think you underestimate: >I can run a name >mapping service and if you already have my key then I can tell you "check >out brandon/megatokyo" and that path will resolve. It still has the same >theoretical problems, I don't think it does. If I have a communications directly from Brandon, let's say I meed Brandon at a party, and let's say I have a basis for believing a correspondence between my face-recognition of Brandon and his fingerprint; then, if he either tells me "brandon/megatokyo" or scribbles it on a piece of paper, I know that the "brandon" is my already established naming root for him, not a name to be looked up in a global or almost global name space. Of course, in a Mission Impossible plot my face and voice recognition of Brandon at the party might be spoofed. But that's the only remaining theoretical problem I see with this use. The only real bad news with subjectivity above is "voice and written" with people you don't already know. For written: I've got my PGP fingerprint on my business card, as do many other people. People already often clip their cards to documents they hand out. This fingerprint is a fine naming root for the naming paths in the document. Xerox has some fine technology for embedding machine readable info unobtrusively in the normal ink printing patterns on paper, and one can imagine a convention for embedding a name-root fingerprint in a company letterhead. For voice: If it's in person, it can still be supplemented by a business card, and often is. If it's not in person, then it's over an electronic medium, which can carry other information as well. Cheers, --MarkM From hal@finney.org Wed Mar 07 03:16:05 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 03:15:32 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id AAA26086 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 00:12:29 -0800 Date: Wed, 7 Mar 2001 00:12:29 -0800 From: hal@finney.org Message-Id: <200103070812.AAA26086@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI Wei's original "three-services model" was like this: > 1. name lookup service: maping user-friendly names/URLs to content-hash > based ids > 2. data location service: maping content-hash based ids to storage servers > (network addresses and data transport protocols) that can deliver the data > 3. data transport service: actually obtaining content from a storage > server given its address and a content-hash based id The current thread focuses on the first service, the one that does name lookups. However this model has (at least) two separate functions for the name lookup, and perhaps it would be useful to split them. One function is providing a "friendly name" for a document; and the other is allowing documents to change without their name changing. This second problem is somewhat specific to the assumption that content hashes are the unforgeable identifiers for documents. With this approach, any time a document is updated, its content hash and therefore its identifier changes. Any content-hash based address will become obsolete once this change has occured. Therefore having the name lookup service is necessary so that you can find the latest version of the document. However there are other ways to handle document revisions. This is part of why Freenet has spawned its multiplicity of key (identifier) classes. Freenet and OceanStore have identifiers which are based on cryptographic keys and signatures. These have the same basic self-verification properties as content hashes but they can allow documents to be updated with the updates signed by the key. If document changes are no longer an issue then the only remaining one is friendly naming. There are a couple of things to note which make this problem less severe. First, only a very small percentage of the web pages on the net presently have names friendly enough that they can be reasonably typed. You might type www.cnn.com to get the headlines, but you won't type http://www.cnn.com/2001/TECH/space/03/06/mir.fungus.reut/index.html to learn about the mutant bacteria threat from Mir. Instead your mail reader will treat it as a link for you, or maybe you'll cut and paste it into your web browser. Given this primitive level of automation, which is already necessary for links to be passed around at all, a link with a 160 bit unforgeable number can look like d787ef5c05460da6cf3d6aa1633655d8861e20f1 in hex or like 14fvXAVGDabPPWqhYzZV2IYeIPE in a base-64 encoding, which is easy to cut and paste; easier in fact than modern URLs which often span more than one line and get forcibly line-wrapped by brain-damaged email software. Second, of course if you are able to use HTML then the link isn't even seen and friendliness does not arise. The remaining case is where people do want to type a URL. This will often be where the URL appears in print, in a magazine or newspaper or billboard; or via some other off-line medium like radio or tv. These URLs have to be extremely short, essentially just one or two words, something that people can remember. This core of the problem remains hard, but note that it will mostly involve businesses and large organizations, and most of the cases will be advertising. Names will typically be some easy-to-remember variant of the company's name, or maybe a clever play on words involving its function. It may well be that for this case a market-based registrar system will work, with just a few large registrars handling the corporate business. You'll have .com, .org and a few others, as we have today. Widely used software will hardwire these popular registries and it will be no easier to overthrow them than it is to get people to ignore the top-level name servers today. The result would be a two-part scheme: a relatively limited and inflexible name system for business trademarks, which identifies the home pages of large organizations; and a completely ad hoc system based on cryptographically self-authorizing, random-looking names for everything else. Hal From blanu@uts.cc.utexas.edu Wed Mar 07 03:48:24 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 03:47:56 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id CAA16104 for ; Wed, 7 Mar 2001 02:47:49 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Wed, 7 Mar 2001 02:47:49 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Yes, I came to this conclusion earlier today. Central authority and multiple competing authorities are really just pet naming paths with an ellided well-known root key. So pet names are really the best it can be. From blanu@uts.cc.utexas.edu Wed Mar 07 03:51:57 2001 Return-Path: Received: from moe.cc.utexas.edu ([128.83.42.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 03:51:42 -0500 Received: from localhost (blanu@localhost) by moe.cc.utexas.edu (8.9.3/8.9.3/cc-uts-client-1.11) with ESMTP id CAA16156 for ; Wed, 7 Mar 2001 02:51:36 -0600 (CST) X-Authentication-Warning: moe.cc.utexas.edu: blanu owned process doing -bs Date: Wed, 7 Mar 2001 02:51:36 -0600 (CST) From: Brandon X-Sender: blanu@moe.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > http://www.cnn.com/2001/TECH/space/03/06/mir.fungus.reut/index.html > > Given this primitive level of automation, which is already necessary > for links to be passed around at all, a link with a 160 bit unforgeable > number can look like d787ef5c05460da6cf3d6aa1633655d8861e20f1 in hex or > like 14fvXAVGDabPPWqhYzZV2IYeIPE in a base-64 encoding, which is easy to > cut and paste; easier in fact than modern URLs which often span more than > one line and get forcibly line-wrapped by brain-damaged email software. The URL is still easier to communicate by voice than then hex or base-64. I think that the importance of a voice-friendly encoding is often overlooked because people underestimate the amount of URL information communicate over the phone and in person. Of course, I don't have any particular ideas on how to convert 160-bit hashes into more voice-friendly encodings, but the least you could do is use hex instead of base-64. From lucas@worldos.com Wed Mar 07 12:26:46 2001 Return-Path: Received: from mail1.panix.com ([166.84.0.212]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 12:26:31 -0500 Received: from obiwan (159-98.nyc.dsl.access.net [166.84.159.98]) by mail1.panix.com (Postfix) with SMTP id 817A448A30 for ; Wed, 7 Mar 2001 12:26:20 -0500 (EST) From: "Lucas Gonze" To: "Global-Scale Distributed Storage Systems" Subject: RE: Naming via SDSI Date: Wed, 7 Mar 2001 12:22:23 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 To get human-friendly IDs, it is not hard to have a namespace provider map names to hashes. IMHO unless the UN runs ICANN there is no solution to the political problem. Always allow the user to specify a namespace (either implicitly or explicitly) and the problem goes away. - Lucas > -----Original Message----- > From: bounce-bluesky-127943@ibiblio.org > [mailto:bounce-bluesky-127943@ibiblio.org]On Behalf Of Brandon > Sent: Wednesday, March 07, 2001 3:52 AM > To: Global-Scale Distributed Storage Systems > Subject: Re: Naming via SDSI > > > > > http://www.cnn.com/2001/TECH/space/03/06/mir.fungus.reut/index.html > > > > Given this primitive level of automation, which is already necessary > > for links to be passed around at all, a link with a 160 bit unforgeable > > number can look like d787ef5c05460da6cf3d6aa1633655d8861e20f1 in hex or > > like 14fvXAVGDabPPWqhYzZV2IYeIPE in a base-64 encoding, which is easy to > > cut and paste; easier in fact than modern URLs which often span more than > > one line and get forcibly line-wrapped by brain-damaged email software. > > The URL is still easier to communicate by voice than then hex or > base-64. I think that the importance of a voice-friendly encoding is often > overlooked because people underestimate the amount of URL information > communicate over the phone and in person. Of course, I don't have any > particular ideas on how to convert 160-bit hashes into more voice-friendly > encodings, but the least you could do is use hex instead of base-64. > > > > --- > You are currently subscribed to bluesky as: lucas@gonze.com > For list information visit http://www.transarc.ibm.com/~ota/bluesky/ > From hal@finney.org Wed Mar 07 13:04:40 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 13:04:30 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id KAA27444 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 10:01:32 -0800 Date: Wed, 7 Mar 2001 10:01:32 -0800 From: hal@finney.org Message-Id: <200103071801.KAA27444@finney.org> To: bluesky@franklin.oit.unc.edu Subject: RE: Naming via SDSI Lucas Gonze, , writes: > To get human-friendly IDs, it is not hard to have a namespace provider map names > to hashes. > > IMHO unless the UN runs ICANN there is no solution to the political problem. > Always allow the user to specify a namespace (either implicitly or explicitly) > and the problem goes away. But now the problem is, how to securely identify a namespace provider, when you supply someone with a URL? Isn't this the same problem? You need either a friendly (user-readable) name for the namespace provider, in which case more than one entity could claim to have that name, or you need a cryptographically secure name, which will be long. Hal From lucas@worldos.com Wed Mar 07 13:36:08 2001 Return-Path: Received: from mail1.panix.com ([166.84.0.212]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 13:35:43 -0500 Received: from obiwan (159-98.nyc.dsl.access.net [166.84.159.98]) by mail1.panix.com (Postfix) with SMTP id D208C489C4 for ; Wed, 7 Mar 2001 13:33:57 -0500 (EST) From: "Lucas Gonze" To: "Global-Scale Distributed Storage Systems" Subject: RE: Naming via SDSI Date: Wed, 7 Mar 2001 13:29:34 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 Hal said: > You need either a friendly (user-readable) name for the namespace > provider, in which case more than one entity could claim to have that > name, or you need a cryptographically secure name, which will be long. I think this boils down to the same insight embodied in webs of trust or SPKI, which is that trust originates at the self. An individual has to make a leap of faith based on the best data they can gather, but from then trusted parties can certify new trusted parties. The bootstrap step you are referring to doesn't have to happen frequently. Let's say there would be a default namespace provider whose function is to map friendly names for namespace providers to hashes. This default namespace provider could be your ISP. On some level you would have to deal with the hash during setup, just like you have to deal with IP settings for a new ISP account. Bootstrap the entire process by getting a public key for your default namespace provider. Verify this in the usual ways, e.g. by looking at the signature in a magazine ad, by calling them on the phone, etc. You can have about as much faith in this as you do in the setup data that your ISP provides. The difference between this and DNS is that trust radiates outward from users, rather than inward from the Root Server in Reston. - Lucas From hal@finney.org Wed Mar 07 13:48:05 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 13:47:51 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id KAA27744 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 10:44:50 -0800 Date: Wed, 7 Mar 2001 10:44:50 -0800 From: hal@finney.org Message-Id: <200103071844.KAA27744@finney.org> To: bluesky@franklin.oit.unc.edu Subject: RE: Naming via SDSI Lucas writes: > The bootstrap step you are referring to doesn't have to happen frequently. > Let's say there would be a default namespace provider whose function is to map > friendly names for namespace providers to hashes. This default namespace > provider could be your ISP. On some level you would have to deal with the hash > during setup, just like you have to deal with IP settings for a new ISP account. What happens if my ISP and your ISP disagree about which namespace provider gets a certain friendly-name, like .com or .biz? If this can't happen, could you explain why? And how does each ISP keep up to date with namespace changes, and decide which namespace provider to honor in the case of disputed name spaces? Many ISPs can barely keep the modems running - will they have the manpower to take on this task? > The difference between this and DNS is that trust radiates outward from users, > rather than inward from the Root Server in Reston. Well, in a sense trust does radiate outward from users in the DNS. The only reason the root servers have power is because of millions of users whose computers consult them to resolve names. See http://www.youcann.org/instructions.html for how to change your computer to consult different roots. If enough people did this, the current root servers would lose power. Hal From lucas@worldos.com Wed Mar 07 14:43:32 2001 Return-Path: Received: from mail1.panix.com ([166.84.0.212]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 14:42:11 -0500 Received: from obiwan (159-98.nyc.dsl.access.net [166.84.159.98]) by mail1.panix.com (Postfix) with SMTP id 968F1487B1 for ; Wed, 7 Mar 2001 14:41:57 -0500 (EST) From: "Lucas Gonze" To: "Global-Scale Distributed Storage Systems" Subject: RE: Naming via SDSI Date: Wed, 7 Mar 2001 14:37:02 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal In-Reply-To: X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 My point here is to say that the need to see hashes instead of friendly names can be limited to setup time. After setup you are either specifying a namespace (based on a chain of trust built out from the root server you have picked) or using the implicit namespace of your root server. For example, your root server might only define four or five name/hash mappings. It's interesting and probably even productive to work on whether it is a good to idea to have a system that permits collisions, but I don't mean to be commenting on that here. Whether your root server is your ISP or some other party is also a different set of issues. I am only addressing the problem of mapping friendly names to hashes. > Well, in a sense trust does radiate outward from users in the > DNS. The only reason the root servers have power is because of > millions of users whose computers consult them to resolve names. > See http://www.youcann.org/instructions.html for how to change your > computer to consult different roots. If enough people did this, the > current root servers would lose power. But there's no way to specify the namespace within a domain reference. Changing the root provider has to be dynamic to be useful. Otherwise you just end up with a different dictator. You could probably hack a solution at the OS level already, at least in Linux... Aside from the mammoth installed base, I don't see why DNS should remain the one true method for mapping friendly names to IPs. Not having white space or punctuation is unnecessarily primitive. Having to have a dot anything is fairly silly as well. All the .foo means is the lookup source - there's no conceptual reason why these things should be mapped on a semantic level like com, biz, aero, edu. That is just a historical artifact from the founding environment of DNS. - Lucas From hal@finney.org Wed Mar 07 15:26:11 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 15:25:57 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id MAA28173 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 12:22:53 -0800 Date: Wed, 7 Mar 2001 12:22:53 -0800 From: hal@finney.org Message-Id: <200103072022.MAA28173@finney.org> To: bluesky@franklin.oit.unc.edu Subject: RE: Naming via SDSI Lucas writes: > My point here is to say that the need to see hashes instead of friendly names > can be limited to setup time. After setup you are either specifying a namespace > (based on a chain of trust built out from the root server you have picked) or > using the implicit namespace of your root server. For example, your root server > might only define four or five name/hash mappings. I'm sorry, I'm still not understanding you. Would your system still handle names like, say, www.microsoft.com? And your point is, that in installing the .com name-server we would use a hash which securely identifies that handler? I don't see how that solves the problem, which is to figure which .com handler we should choose, and more to the point, how to be sure that the .com handler you chose is the same one I use. Or putting it another way, what do you do when two namespace resolvers claim to handle the same namespace? It doesn't help us to have a secure hash to the namespace resolver. That's not the hard part of the problem. One solution is to not have friendly-names for namespaces. In other words, TLDs would be hashes. You'd have www.microsoft.14fvXAVGDabPPWqhYzZV2IYeIPE. That last string would unambiguously designate a namespace resolver; there would be no disputes possible, and everyone would use the same resolver. The question is, can we get this level of clarity and security while keeping friendly names for the namespaces. Hal From lucas@worldos.com Wed Mar 07 16:27:27 2001 Return-Path: Received: from mail2.panix.com ([166.84.0.213]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 16:27:10 -0500 Received: from obiwan (159-98.nyc.dsl.access.net [166.84.159.98]) by mail2.panix.com (Postfix) with SMTP id E379D9CE5 for ; Wed, 7 Mar 2001 16:27:03 -0500 (EST) From: "Lucas Gonze" To: "Global-Scale Distributed Storage Systems" Subject: RE: Naming via SDSI Date: Wed, 7 Mar 2001 16:21:15 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 In-Reply-To: > Would your system still > handle names like, say, www.microsoft.com? And your point is, that > in installing the .com name-server we would use a hash which securely > identifies that handler? I don't see how that solves the problem, which > is to figure which .com handler we should choose, and more to the point, > how to be sure that the .com handler you chose is the same one I use. I would add one more level of indirection, to a namespace provider that maps the name of the .com namespace provider to the hash of the .com namespace provider. I would eliminate semantic landgrabs for concepts like ".com" and give namespace providers proper names. So www.microsoft.com.realnames.myRootProvider is the full path. myRootProvider is a hash/name mapping I entered manually, realnames is a hash/name mapping provided by myRootProvider, com is a mapping provided by realnames, etc. As you move from right to left the likelyhood of a collision would go up. myRootProvider must be canonical to the best of my ability to make it so. realnames should be very dependable. com would be somewhat disputable. microsoft would be fairly disputable. The possibility of collisions is a good thing. For the most part there won't be collisions, because it's in everyone's interest to share most of this data. But if there are conflicts between namespace providers just let users vote with their feet. > One solution is to not have friendly-names for namespaces. In other words, > TLDs would be hashes. You'd have www.microsoft.14fvXAVGDabPPWqhYzZV2IYeIPE. > That last string would unambiguously designate a namespace resolver; there > would be no disputes possible, and everyone would use the same resolver. quite cool! Ok, so add another level of indirection to get www.microsoft.com.realnames.14fvXAVGDabPPWqhYzZV2IYeIPE. and we are on more or less the same turf, except that I would allow for gradations of disputability. - Lucas From hal@finney.org Wed Mar 07 17:22:30 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 17:22:19 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id OAA28801 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 14:19:18 -0800 Date: Wed, 7 Mar 2001 14:19:18 -0800 From: hal@finney.org Message-Id: <200103072219.OAA28801@finney.org> To: bluesky@franklin.oit.unc.edu Subject: RE: Naming via SDSI Lucas writes: > I would add one more level of indirection, to a namespace provider that maps the > name of the .com namespace provider to the hash of the .com namespace provider. > I would eliminate semantic landgrabs for concepts like ".com" and give namespace > providers proper names. So www.microsoft.com.realnames.myRootProvider is the > full path. myRootProvider is a hash/name mapping I entered manually, realnames > is a hash/name mapping provided by myRootProvider, com is a mapping provided by > realnames, etc. In what format would you (or, more to the point, Microsoft) publish this name? Is it literally in this form, www.microsoft.com.realnames.myRootProvider? What does the string myRootProvider mean to someone else? And what is "realnames" here? That is a namespace provider, right? What happens if two companies both claim to be responsible for the "realnames" mapping? Does each different ISP decide which one to trust? > As you move from right to left the likelyhood of a collision would go up. > myRootProvider must be canonical to the best of my ability to make it so. > realnames should be very dependable. com would be somewhat disputable. > microsoft would be fairly disputable. What do you mean by "collisions"? Is it that the .com provider might give out two different mappings for microsoft.com? Surely it would not do this. Or are you worried about two different .com providers, both giving out microsoft.com? If so, then surely realnames wouldn't have given out two different mappings for .com, right? So there must have been two different realnames providers, both giving out mappings for .com. On this basis, the only way I see collisions occuring is from the highest level, if two different people are using different root providers, and so they might have different .realnames providers, different .com providers, and so on. But this would be a collision at the right, and would not increase in probability as you go to the left. So I think I am misunderstanding you. > The possibility of collisions is a good thing. For the most part there won't be > collisions, because it's in everyone's interest to share most of this data. But > if there are conflicts between namespace providers just let users vote with > their feet. This was the suggestion made earlier, that we would just let people compete for TLDs and let the market decide. I gave reasons [1] why I thought this was a bad idea. I thought you were offering an alternative to this. Maybe I was mistaken and you had the same idea all along. [1] http://www.mail-archive.com/bluesky%40franklin.oit.unc.edu/msg00088.html > > One solution is to not have friendly-names for namespaces. In other words, > > TLDs would be hashes. You'd have www.microsoft.14fvXAVGDabPPWqhYzZV2IYeIPE. > > That last string would unambiguously designate a namespace resolver; there > > would be no disputes possible, and everyone would use the same resolver. > > quite cool! Ok, so add another level of indirection to get > www.microsoft.com.realnames.14fvXAVGDabPPWqhYzZV2IYeIPE. The problem with this solution is that it is not a friendly name. The goal of this discussion is to see if there is a way to have TLD names be friendly. Hal From md98-osa@nada.kth.se Wed Mar 07 17:30:14 2001 Return-Path: Received: from mail1.telia.com ([195.67.239.85]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 17:30:00 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14amTm-0000EE-00 for bluesky@franklin.oit.unc.edu; Wed, 07 Mar 2001 23:32:22 +0100 Date: Wed, 7 Mar 2001 23:32:22 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010307233222.A644@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 12:22:53PM -0800 On Wed, Mar 07, 2001 at 12:22:53PM -0800, hal@finney.org wrote: <> > One solution is to not have friendly-names for namespaces. In other words, > TLDs would be hashes. You'd have www.microsoft.14fvXAVGDabPPWqhYzZV2IYeIPE. > That last string would unambiguously designate a namespace resolver; there > would be no disputes possible, and everyone would use the same resolver. > The question is, can we get this level of clarity and security while keeping > friendly names for the namespaces. I don't think so. After all, a multilayered name can be viewed as relative path along a graph. In order for two parties to follow such a path and end up at the same point, they have to start at the same point. For them to start at the same point, then they either need to go by convention, or the absolute identity of the starting point has to be given. Making the absolute value "nice" is (AFAIK) impossible with current PK crypto - and probably will remain since a PK algo where the public values are chosen and the private values are derived from them is impossible (what is to stop a second party from choosing the same PK value). And even if it wasn't impossible, it would make the whole layered name thing rather pointless. So that leaves convention, and convention unavoidably means both centralization of the service "root" and politics. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From hal@finney.org Wed Mar 07 18:45:58 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 18:45:26 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id PAA29302 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 15:42:25 -0800 Date: Wed, 7 Mar 2001 15:42:25 -0800 From: hal@finney.org Message-Id: <200103072342.PAA29302@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI Oskar Sandberg writes: > On Wed, Mar 07, 2001 at 12:22:53PM -0800, hal@finney.org wrote: > > The question is, can we get this level of clarity and security while keeping > > friendly names for the namespaces. > > I don't think so. After all, a multilayered name can be viewed as relative > path along a graph. In order for two parties to follow such a path and end > up at the same point, they have to start at the same point. For them to > start at the same point, then they either need to go by convention, or the > absolute identity of the starting point has to be given. Yes, that seems very true. > Making the absolute value "nice" is (AFAIK) impossible with current PK > crypto - and probably will remain since a PK algo where the public values > are chosen and the private values are derived from them is impossible > (what is to stop a second party from choosing the same PK value). And even > if it wasn't impossible, it would make the whole layered name thing rather > pointless. So that leaves convention, and convention unavoidably means > both centralization of the service "root" and politics. I agree. Others have proposed alternatives to politics in market-based mechanisms, but markets have trouble negotiating monopolies. There's an economic principle called "rent dissipation" in which companies will expend almost the total value of a rent-producing asset in order to acquire it. This is wasteful and inefficient. Hal From lucas@worldos.com Wed Mar 07 19:34:42 2001 Return-Path: Received: from mail2.panix.com ([166.84.0.213]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 19:34:21 -0500 Received: from obiwan (159-98.nyc.dsl.access.net [166.84.159.98]) by mail2.panix.com (Postfix) with SMTP id 12B4A95BA for ; Wed, 7 Mar 2001 18:23:28 -0500 (EST) From: "Lucas Gonze" To: "Global-Scale Distributed Storage Systems" Subject: RE: Naming via SDSI Date: Wed, 7 Mar 2001 18:16:14 -0500 Message-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook IMO, Build 9.0.2416 (9.0.2910.0) Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.00.2615.200 In-Reply-To: > On this basis, ...this is the right interpretation of what I mean by collision. > the only way I see collisions occuring is from the highest level I think that we're agreeing and using different language. Collisions are caused on the right, but are visible on the left. Once a collision happens (which happens as you move left), there are now different parties serving a namespace. I'd add one refinement here, which is that collisions could die out if, e.g., the two different realnames providers agree on a mapping for .com. > This was the suggestion made earlier, that we would just let people > compete for TLDs and let the market decide. I gave reasons [1] why I > thought this was a bad idea. I thought you were offering an alternative > to this. Maybe I was mistaken and you had the same idea all along. > > [1] http://www.mail-archive.com/bluesky%40franklin.oit.unc.edu/msg00088.html I read it. I agree that there are ambiguity, uncertainty and costs in doing name lookups, and that is not ideal. Where I differ is that I think that the question is not whether conflict is allowed but how to minimize the damage. It's hard to believe that China, Libya and the US will always be able to resolve conflicts. At a certain size of the Internet you have to find ways to agree to disagree. So my suggestion re: bootstrapping a friendly name mapping service is intended to make the best of a bad situation, not to eliminate it, because I don't think it can be eliminated. - Lucas From md98-osa@nada.kth.se Wed Mar 07 19:41:46 2001 Return-Path: Received: from mail1.telia.com ([62.20.149.144]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 19:41:10 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14aoWk-0000O8-00 for bluesky@franklin.oit.unc.edu; Thu, 08 Mar 2001 01:43:34 +0100 Date: Thu, 8 Mar 2001 01:43:34 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Naming via SDSI Message-ID: <20010308014334.B1364@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 03:42:25PM -0800 On Wed, Mar 07, 2001 at 03:42:25PM -0800, hal@finney.org wrote: > Oskar Sandberg writes: > > On Wed, Mar 07, 2001 at 12:22:53PM -0800, hal@finney.org wrote: > > > The question is, can we get this level of clarity and security while keeping > > > friendly names for the namespaces. > > > > I don't think so. After all, a multilayered name can be viewed as relative > > path along a graph. In order for two parties to follow such a path and end > > up at the same point, they have to start at the same point. For them to > > start at the same point, then they either need to go by convention, or the > > absolute identity of the starting point has to be given. > > Yes, that seems very true. One hypothetical model for certain circumstances could be make this graph a two-way such, so that two people could negoiate a starting point, and go from there based on reversing their path to it. Something like: Alice: You can get a Free dvd player at my site, fuckwaresux.truth Bob: But I thought ICANN had decided .truth was solely the domain of large media? Alice: Yeah, but I'm not using ICANN's root. Do you know microsoft's webpage under ICANN? Bob: Yeah, it's microsoft.com of course. Alice: Well, it's microsoft.criminal.orgs here, so you should find my site at fuckwaresux.truth.orgs.criminal.microsoft.com (from ICANN's root -> ICANN's com tld -> microsoft's site -> Alice's criminal.orgs domain -> Alice's orgs tld -> Alice's root -> Alice's truth tld -> the Fuckware (Futile Unnecessary Control-Keeping softWARE) Sucks site.) Of course, this doesn't work for several reasons. Firstly, it's pretty useless for tv ads and billboards where there is no room for negoiation, and that is where it is needed. Secondly, there is really no such thing as a two way link in cyberspace - the chances that microsoft would actually have a reverse entry to criminal.orgs isn't exactly very large. Thirdly, it assumes you trust every step along the route, which is pretty nuts. > > Making the absolute value "nice" is (AFAIK) impossible with current PK > > crypto - and probably will remain since a PK algo where the public values > > are chosen and the private values are derived from them is impossible > > (what is to stop a second party from choosing the same PK value). And even > > if it wasn't impossible, it would make the whole layered name thing rather > > pointless. So that leaves convention, and convention unavoidably means > > both centralization of the service "root" and politics. > > I agree. Others have proposed alternatives to politics in market-based > mechanisms, but markets have trouble negotiating monopolies. There's an > economic principle called "rent dissipation" in which companies will > expend almost the total value of a rent-producing asset in order to > acquire it. This is wasteful and inefficient. I considers markets to be politics. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From adam@cypherpunks.ai Wed Mar 07 21:16:45 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 21:16:30 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 059F54C; Wed, 7 Mar 2001 22:16:15 -0400 (AST) Date: Wed, 7 Mar 2001 22:16:15 -0400 From: Adam Back To: bluesky@franklin.oit.unc.edu Subject: distributed immutable namespace Message-ID: <20010307221615.A6287@cypherpunks.ai> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i Return-Path: Sender: adam@cypherpunks.ai I proposed on the crypto lists a few years back an idea about namespaces to create a censor resistant namespace. A namespace is a mapping from a human readable name into some bit string (eg. ipv4 IP address, content hash, location handle, etc). The idea is to try to create a global namespace collaboratively maintained by servers which audit each others behavior and compensate for rogue behavior in collaboration with client name server protocols. The aim is to create a namespace where Names are strictly first-come first served and persistent. Only the author who registers a key at the time he reserves the name is able to change the mapping. Local censorship and revisionism is allowed, but only in local views of the root namespace. Users browsers may be configured (or even come pre-configured) to censor local names in the domain they are distributed in. (Eg. a .cn filter, or a .us filter). But anyone should be able to get to the root unmodified namespace, either by disabling the filters, or by using some syntax which indicates that the name is in the root immutable namespace. There are two protocols involved: 1) Public Auditability - Use Merkle Authentication Trees [*] to prevent revisionism on the part of namespace servers. Namespace servers publish master hashes, users and namespace servers audit each others adherence to the global immutable root policy. 2) Revision Recovery - The client software should include protocols to accept notices from other namespace servers that detect a rogue namespace server. rogue namespace servers responses pass through the revision recovery protocol which works out if the repsonse is from a modified part of the Authentication Tree, and if so recovers the necessary original values from other servers. Servers cache the original values of modified sections of rogue servers Authentication Trees. The combination of protocols in clients and servers ensures that users can choose which filters they want to subscribe to. Companies may be satisfied with the subset of names that survive the policies of a monopoly unaudited root server. People who want more robust names can use the root namespace explicitly. For efficiency you probably want namespace servers to do the auditing. You don't want to audit every lookup back to the root even thought it is relatively efficient. Users can of course independently audit also. Adam [*] A Merkle Authentication Tree is basically a tree of hashes to allow the efficient verification of a particular leaf in the tree with log(n) queries. Imagine a binary tree representing a namespace with names as the leaf nodes. The parent of a leaf node is the hash of the pair of names below it. The parent of that node with it's neighbor is the hash of it and the neighbor hash, and so on up to the root which is the master hash of all the hash of hashes of names down to the leaves. Fortunately Merkle's patent has expired. From hal@finney.org Wed Mar 07 21:25:48 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 21:25:33 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id SAA00621 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 18:22:35 -0800 Date: Wed, 7 Mar 2001 18:22:35 -0800 From: hal@finney.org Message-Id: <200103080222.SAA00621@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: distributed immutable namespace Adam Back writes: > The idea is to try to create a global namespace > collaboratively maintained by servers which audit each > others behavior and compensate for rogue behavior in > collaboration with client name server protocols. > > The aim is to create a namespace where Names are strictly > first-come first served and persistent. Only the author > who registers a key at the time he reserves the name is > able to change the mapping. That's an interesting idea, but isn't there a danger that a first-come first-served policy will result in someone writing a script to reserve all one-letter, two-letter, three-letter words etc. as fast as he can generate keys? Hal From orasis@acm.cs.umn.edu Wed Mar 07 21:35:17 2001 Return-Path: Received: from acm.cs.umn.edu ([128.101.32.12]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 21:35:11 -0500 Received: from go.cs.umn.edu (go.cs.umn.edu [128.101.32.4]) by acm.cs.umn.edu (8.11.2/8.11.0) with ESMTP id f282dEF52275 for ; Wed, 7 Mar 2001 20:39:14 -0600 (CST) (envelope-from orasis@acm.cs.umn.edu) Received: (from orasis@localhost) by go.cs.umn.edu (8.11.2/8.11.0) id f27KaBi36620 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 20:36:11 GMT (envelope-from orasis) Date: Wed, 7 Mar 2001 20:36:11 +0000 From: Justin Chapweske To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010307203610.C33945@go.cs.umn.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 06:22:35PM -0800 > > That's an interesting idea, but isn't there a danger that a first-come > first-served policy will result in someone writing a script to reserve > all one-letter, two-letter, three-letter words etc. as fast as he can > generate keys? > You could slow it down with a "hash cash" type protocol where a person would have to for example burn $100 worth of CPU cycles in order to register a top(ish) level domain. This is currently how the real DNS system works....maybe a very top level domain would cost $100M CPU cycles to register to avoid a completely flat name space. I love having these conversations on public mailing lists because you know some bastard would patent these pretty obvious ideas otherwise. -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From adam@cypherpunks.ai Wed Mar 07 21:43:33 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 21:43:28 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 471BA4C; Wed, 7 Mar 2001 22:43:21 -0400 (AST) Date: Wed, 7 Mar 2001 22:43:21 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010307224321.B6287@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 06:22:35PM -0800 Return-Path: Sender: adam@cypherpunks.ai It would seem that someone could DoS the namespace as you describe. It sounds like this attack would also work on other namespace proposals, so is perhaps fairly general. What could we do about it? Charge real ecash would be a satisfying answer, but then we have boot strap problems, as there isn't one yet, and global e-cash systems robust against being themselves censored are probably an even harder problem. Perhaps you could use hashcash to pay for your name. name creations are infrequent enough that users could put up with involve a reasonably expensive computation. Users of names could also submit hashcash on the names they use so that popular names get higher valued hashcash. Also you could probably retain auditability if you allowed policies of dropping the lowest valued names in event of a flood. None of this prevents attacks, just makes them a little more expensive. An is-a-person credential and a limited quota per-person might be nice too, but that has it's own boot strap problems, and imports trust on the credential issuer. Adam On Wed, Mar 07, 2001 at 06:22:35PM -0800, hal@finney.org wrote: > Adam Back writes: > > The idea is to try to create a global namespace > > collaboratively maintained by servers which audit each > > others behavior and compensate for rogue behavior in > > collaboration with client name server protocols. > > That's an interesting idea, but isn't there a danger that a first-come > first-served policy will result in someone writing a script to reserve > all one-letter, two-letter, three-letter words etc. as fast as he can > generate keys? From hal@finney.org Wed Mar 07 22:16:13 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 22:16:01 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id TAA01318 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 19:13:01 -0800 Date: Wed, 7 Mar 2001 19:13:01 -0800 From: hal@finney.org Message-Id: <200103080313.TAA01318@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: distributed immutable namespace Justin Capweske writes: > You could slow it down with a "hash cash" type protocol where a person > would have to for example burn $100 worth of CPU cycles in order to > register a top(ish) level domain. This is currently how the real DNS > system works....maybe a very top level domain would cost $100M CPU cycles > to register to avoid a completely flat name space. Hashcash at this scale makes me wince; all those wasted cycles... Verisign bought Network Solutions for 21 BILLION dollars. That gave them a big chunk of .com, .org and .net. Can you imagine burning through that much hashcash? Hashcash is rent dissipation in action. It doesn't make economic sense to sell assets for hashcash. Better to set up some kind of global fund and let buyers pay real money, then distribute the proceeds to everyone on earth. Or maybe use the money to reduce global warming, or to save the whales. Anything is better than flushing the money down the toilet, which is what hashcash does. Hal From pete@retep.yi.org Wed Mar 07 22:46:24 2001 Return-Path: Received: from femail2.rdc1.on.home.com ([24.2.9.89]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 22:46:19 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail2.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010308034604.LZXP606.femail2.rdc1.on.home.com@gw.localdomain> for ; Wed, 7 Mar 2001 19:46:04 -0800 Received: from petes.localdomain ([192.168.1.1] ident=mail) by gw.localdomain with esmtp (Exim 3.22 #1 (Debian)) id 14arNS-0001TF-00 for ; Wed, 07 Mar 2001 22:46:10 -0500 Received: from pete by petes.localdomain with local (Exim 3.12 #1 (Debian)) id 14arNR-0000zF-00 for ; Wed, 07 Mar 2001 22:46:09 -0500 Date: Wed, 7 Mar 2001 22:46:09 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010307224609.B3677@petes.localdomain> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="1LKvkjL3sHcu1TtY" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 07:13:01PM -0800 Sender: Peter Todd --1LKvkjL3sHcu1TtY Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 07, 2001 at 07:13:01PM -0800, hal@finney.org wrote: > Hashcash at this scale makes me wince; all those wasted cycles... You know I heard a estimites that Seti@Home burns 18tons of fossile fuel a *minute* Hashcash would do the same thing. > Verisign bought Network Solutions for 21 BILLION dollars. That gave > them a big chunk of .com, .org and .net. Can you imagine burning > through that much hashcash? >=20 > Hashcash is rent dissipation in action. It doesn't make economic sense > to sell assets for hashcash. What do you mean by "rent dissipation in action"? The big problem is hashcash is inflation. Mores law plus vastly increasing supply. We'll end up with insanely high inflation rates if we're not carefull, at least 100% every 18 months. (mores law) Probably a few hundred times higher in practice. Inflation causes another problem. Everything dealing with hashcash must be able to automaticly adjust prices in realtime. If we end up with 10% a day inflation rates, not impossible by any means, we're going to have to do that. And finally inflation may become such a problem that we outgrow the storage space available to store all this hashcash Compound interest is a wonderfull thing you know... > Better to set up some kind of global fund and let buyers pay real money, > then distribute the proceeds to everyone on earth. Or maybe use the > money to reduce global warming, or to save the whales. Anything is better > than flushing the money down the toilet, which is what hashcash does. But hashcash doesn't need any centralized authority to work. Find something else that doesn't. --1LKvkjL3sHcu1TtY Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.1 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6pwCBLdZCJKx+dn0RAvO4AJ9OgMSx8Oe7cHSghrQ7PomugDq9vwCeOIdw dGiS7u56EH2EcMDFjDaSt04= =Z/6Y -----END PGP SIGNATURE----- --1LKvkjL3sHcu1TtY-- From hal@finney.org Wed Mar 07 23:02:46 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 23:02:35 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id TAA04178 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 19:59:30 -0800 Date: Wed, 7 Mar 2001 19:59:30 -0800 From: hal@finney.org Message-Id: <200103080359.TAA04178@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: distributed immutable namespace Peter Todd writes: > On Wed, Mar 07, 2001 at 07:13:01PM -0800, hal@finney.org wrote: > > Hashcash is rent dissipation in action. It doesn't make economic sense > > to sell assets for hashcash. > > What do you mean by "rent dissipation in action"? Let's suppose that of the $21 billion that Network Solutions went for, half was for .com, so it is worth $10 billion. Then when we make .com available to the highest hashcash bidder, it makes sense for potential bidders to waste up to $10 billion in cycles in order to generate their hashcash. Anyone who wastes less than this won't get it. Therefore there will be $10 billion wasted in order to acquire an asset worth $10 billion. Net benefit to society: zero. Whereas if the $10 billion had gone into a fund to do something useful, there would be a net positive benefit to society. > > Better to set up some kind of global fund and let buyers pay real money, > > then distribute the proceeds to everyone on earth. Or maybe use the > > money to reduce global warming, or to save the whales. Anything is better > > than flushing the money down the toilet, which is what hashcash does. > > But hashcash doesn't need any centralized authority to work. Find > something else that doesn't. Well, this could be a DEcentralized authority, similar to what Adam was proposing. But authority it is. Hal From orasis@acm.cs.umn.edu Wed Mar 07 23:45:03 2001 Return-Path: Received: from acm.cs.umn.edu ([128.101.32.12]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 23:44:52 -0500 Received: from go.cs.umn.edu (go.cs.umn.edu [128.101.32.4]) by acm.cs.umn.edu (8.11.2/8.11.0) with ESMTP id f284mtF55311 for ; Wed, 7 Mar 2001 22:48:55 -0600 (CST) (envelope-from orasis@acm.cs.umn.edu) Received: (from orasis@localhost) by go.cs.umn.edu (8.11.2/8.11.0) id f27MjrA36850 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 22:45:53 GMT (envelope-from orasis) Date: Wed, 7 Mar 2001 22:45:53 +0000 From: Justin Chapweske To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010307224553.D33945@go.cs.umn.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from adam@cypherspace.org on Wed, Mar 07, 2001 at 10:43:21PM -0400 God I love synchronicity (same ideas at the same time). > > Perhaps you could use hashcash to pay for your name. name > creations are infrequent enough that users could put up > with involve a reasonably expensive computation. Users of > names could also submit hashcash on the names they use so > that popular names get higher valued hashcash. > -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From orasis@acm.cs.umn.edu Wed Mar 07 23:48:56 2001 Return-Path: Received: from acm.cs.umn.edu ([128.101.32.12]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 23:48:41 -0500 Received: from go.cs.umn.edu (go.cs.umn.edu [128.101.32.4]) by acm.cs.umn.edu (8.11.2/8.11.0) with ESMTP id f284qfF55405 for ; Wed, 7 Mar 2001 22:52:41 -0600 (CST) (envelope-from orasis@acm.cs.umn.edu) Received: (from orasis@localhost) by go.cs.umn.edu (8.11.2/8.11.0) id f27Mndc36860 for bluesky@franklin.oit.unc.edu; Wed, 7 Mar 2001 22:49:39 GMT (envelope-from orasis) Date: Wed, 7 Mar 2001 22:49:39 +0000 From: Justin Chapweske To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010307224939.E33945@go.cs.umn.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 07:13:01PM -0800 > > Hashcash at this scale makes me wince; all those wasted cycles... > Very good point. So what we really need is a non-profit DNS where the non-profit organization technically has no control over the DNS system. This is theoretically possible? -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From adam@cypherpunks.ai Wed Mar 07 23:49:20 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 07 Mar 2001 23:48:58 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 4A67B52; Thu, 8 Mar 2001 00:48:51 -0400 (AST) Date: Thu, 8 Mar 2001 00:48:51 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308004851.A35061@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from hal@finney.org on Wed, Mar 07, 2001 at 07:59:30PM -0800 Return-Path: Sender: adam@cypherpunks.ai On Wed, Mar 07, 2001 at 07:59:30PM -0800, hal@finney.org wrote: > > > Better to set up some kind of global fund and let buyers pay real money, > > > then distribute the proceeds to everyone on earth. Or maybe use the > > > money to reduce global warming, or to save the whales. Anything is better > > > than flushing the money down the toilet, which is what hashcash does. > > > > But hashcash doesn't need any centralized authority to work. Find > > something else that doesn't. > > Well, this could be a DEcentralized authority, similar to what Adam > was proposing. But authority it is. You would want the ability to buy names strongly anonymously; so you want a payer anonymous ecash system which you can pay for with credit cards, or perhaps with checks as credit cards have too high a fraud rate to risk selling strongly anonymous ecash for. In the short term the payment system would be the system weak point because it's an identifiable central server. Censors would close down the mint, much as the RIAA is having an easy job getting courts to rule that Napster's central servers have to shut down. Even though ecash to pay for names in a namespace is several steps removed from content some of which RIAA may dispute, I doubt a court will feel this any reason not to rule that the mint be shut down. Perhaps there is even a precedent of sorts in the MPAA decision ruling that links to DeCSS source code should be removed. Either way we have a problem with DoS against both the namespace and the distributed content store itself. It may be interesting to investigate how well the name resolution, auditability and revision recovery protocols can be made to scale with number of name servers and percentage of modified values. Hashcash may burn resources, but it appears to be all that we have in the way of distributed DoS resistance. Also hashcash would typically burn resources that would be revving away idle anyway (modulo the difference in power saving state some hardware may switch to if the processor is idle). Unless it got really heated and people started building expensive custom hardware purely to escalate the race on desirable namespace. Another risk for hashcash apart from custom hardware races is virii -- imagine a virus or worm with a payload which computed hashcash collisions. If names are richer there may be less pressure to land grab names to re-sell. Internet domains are basically a single word in .{com,net,org}; if the name space had an arbitrary Top Level Domains and 2nd level domain like namespace it would be easier to find nice sounding names. Whether or not owning a names gives ownership of all names which contain it as a left substring is another question. In eternity USENET http://foo.eternity/ could have completely different ownership to http://foo.eternity/bar/, and there was no support for a link between them. The ownership of foo.eternity just gave the ability to choose which sub-URLs to link from the index.html file. Other people could have sub-URLs which you don't link to (or even which you do), either way authorship signatures describe authorship, and hypertext references provide links. I did think eternity USENET style names might be less vulnerable to land grabbing because the owner of the virtual domain name doesn't automatically own URLs under that. But perhaps that's undesirable anyway because it violates expectations that the URLs which look like they are on the same virtual domain are somehow under related ownership. Adam From md98-osa@nada.kth.se Thu Mar 08 08:04:07 2001 Return-Path: Received: from mail1.telia.com ([195.67.239.123]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 08:03:54 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14b07X-0000M3-00 for bluesky@franklin.oit.unc.edu; Thu, 08 Mar 2001 14:06:19 +0100 Date: Thu, 8 Mar 2001 14:06:19 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308140619.G800@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from adam@cypherspace.org on Wed, Mar 07, 2001 at 10:16:15PM -0400 On Wed, Mar 07, 2001 at 10:16:15PM -0400, Adam Back wrote: <> > There are two protocols involved: > > 1) Public Auditability - Use Merkle Authentication Trees > [*] to prevent revisionism on the part of namespace > servers. Namespace servers publish master hashes, users > and namespace servers audit each others adherence to the > global immutable root policy. Sorry if I'm being stupid here, but what exactly is the tree authenticating? It seems to me that MATs need to be static, which a namespace would obviously not be. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From adam@cypherpunks.ai Thu Mar 08 13:10:52 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 13:10:39 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id 92B8859; Thu, 8 Mar 2001 14:10:28 -0400 (AST) Date: Thu, 8 Mar 2001 14:10:28 -0400 From: Adam Back To: Oskar Sandberg , Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308141028.A84590@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from Oskar Sandberg on Thu, Mar 08, 2001 at 02:06:19PM +0100 Return-Path: Sender: adam@cypherpunks.ai On Thu, Mar 08, 2001 at 02:06:19PM +0100, Oskar Sandberg wrote: > > 1) Public Auditability - Use Merkle Authentication Trees > > [*] to prevent revisionism on the part of namespace > > servers. Namespace servers publish master hashes, users > > and namespace servers audit each others adherence to the > > global immutable root policy. > > Sorry if I'm being stupid here, but what exactly is the tree > authenticating? It seems to me that MATs need to be static, which a > namespace would obviously not be. You publish a list of master hashes, say one per day, each hashing the days master and the previous days master hash. If you care you can get the server to commit to names blind so that you can then prove it excluded you. (eg. hash name you would submit with random nonce, get server to sign that random number hashed with todays master hash, then submit the real name and nonce so the server can verify. If the name does not show up in the next days hash tree, you have proof that it excluded you in the form of the signed receipt, on presentation of that receipt the server will either add you to the current tree or be considered t ohave cheated. Adam From ian@technic.uprizer.com Thu Mar 08 13:26:08 2001 Return-Path: Received: from technic.uprizer.com ([4.18.49.34]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 13:26:02 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f28IWov02305 for bluesky@franklin.oit.unc.edu; Thu, 8 Mar 2001 10:32:50 -0800 Date: Thu, 8 Mar 2001 10:32:50 -0800 From: Ian Clarke To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308103250.E1618@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="Rgf3q3z9SdmXC6oT" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from adam@cypherspace.org on Wed, Mar 07, 2001 at 10:43:21PM -0400 --Rgf3q3z9SdmXC6oT Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Wed, Mar 07, 2001 at 10:43:21PM -0400, Adam Back wrote: > Perhaps you could use hashcash to pay for your name. name > creations are infrequent enough that users could put up > with involve a reasonably expensive computation. Users of > names could also submit hashcash on the names they use so > that popular names get higher valued hashcash. This same issue came up on the Freenet mailing list a while back. We weren't happy with hashcash since it would make your ability to publish information proportional to the speed of your PC (which is roughly proportional to the size of your wallet), and we didn't like that at a philisophical level. I suggested a concept called "Think Cash", here is a copy of a document I wrote to describe the idea (note that this is really just an exploration of the problem more than a solution): -- One of the core problems with the Internet today, and its open trusting nature, is that of Denial of Service or DOS attacks. Basically, this is the computer equivalent of making it impossible for someone to use their phone by phoning them once a minute, 24 hours a day, 365 days a year automatically. Of course, with phones, people who did this would be cut off very quickly, however this is not so easy with the Internet. Further, tools such as Freenet which aim to protect people's anonymity, could compound this problem even more in some circumstances. (It is worth noting that Freenet's architecture does make DOS attacks less damaging by avoiding centralized control). One obvious example of DOS attacks is "Spam" or unsolicited email. Here people send out thousands or millions of emails to people, generally advertising a product or service that only a tiny fraction of the recepients will be interested in. Ways to address the problem 1. Increased regulation One approach is increased regulation of the Internet, making it easier to detect and remove those mounting DOS attacks. The problem with this is that many people (myself included) feel that the real value of the Internet is as a forum for free speech, and the tools developed to counter DOS attacks could easily be turned against other Internet users to censor them. In this case, the cure would be much worse than the disease. 2. Hash cash The second approach is to make it too costly for people to mount a DOS attack, but where the cost of normal internet usage remains insignificant. Hash cash is one solution where a user is forced to perform a time-consuming (and therefore money consuming) calculation, before submitting something to the system (such as sending an email or transmitting an IP packet to a given IP address). This is quite straight-forward to implement, however its usefulness is unclear. Firstly it would create a bias against users of slower computers. Secondly, even if it took 1 minute to perform the calculation nescessary to send an email, you could still send a considerable amount of email in a 24 hour period. 3. Think cash A more sophisticated idea, which (as far as I am aware) has not been considered prior to my suggesting it on one of the Freenet Project mailing lists several months ago, is to force ``a human`` to do a small amount of work, which only a human could do, before submitting each piece of information. The aim is to make it impossible to automate the task. Thinking of a way to do this is not easy - any such mechanism would need the following properties: 1. Only a human should be able to perform the task or test 2. A computer should be able to create a test automatically 3. A computer should be able to judge the results of the test Some possible approaches There are a number of general areas where we could look for such a test. Of particular interest to us should be tasks which humans find easy, but which machines find difficult. Examples include image recognition and speech recognition. The problem here is not so much creating the tests (requirement 2), but judging them (requirement 3). There is the further constraint that each test must have enough possible answers so that a computer could not simply attempt all of them. Testing for intelligence One particular line of enquiry which caught my attention was the "Turing Test". Alan Turing was a British cryptographer who helped to break German cyphers during the second world war. He also had a keen interest in the concept of machine intelligence, and in particular, how to test a machine for intelligence. His idea, what we now call the Turing test, was clever in that it largely evaded the question of what intelligence actually was. A simplified version of the actual test would work like this. Someone would have a conversation (using a keyboard and screen, much like IRC) with what could be either another person, or a computer pretending to be a person. They would have to decide whether they were talking to a human or a machine. This test would be repeated, and if they got it wrong around half of the time, then the machine could be regarded as being intelligent. This approach may suggest a solution to the dilemma outlined above. The seeds of a solution I started to think about whether it would be possible to create a system where all the users, while being tested by the system, were actually testing each-other too. Each user might be asked to provide an answer to a question which will demonstrate their intelligence, to provide a question which will test someone elses intelligence, and to give an opinion as to the answers given by a number of other people to several questions. Failure to do any of these things to the best of their ability may result in their submission not being accepted. Remaining issues Of course, there are still many details to be worked out, how can the above system be built so as to make abuse impossible? What if a computer made so many submissions that it saturated the system, and could validate its own submissions as being correct? None the less, at least this suggests that a robust think-cash mechanism is a possibility. As the threat of malicious attacks on the Internet get more and more serious, people may be forced to adopt a mechanism such as this.=20 --Rgf3q3z9SdmXC6oT Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6p9BSQtgxRWSmsqwRAgz2AJ97uGMuTPran+XeFVmTjgg3H4dINgCdHgl3 yYncPBh/3l8P8lavFfCcvYM= =w8Fr -----END PGP SIGNATURE----- --Rgf3q3z9SdmXC6oT-- From hal@finney.org Thu Mar 08 13:44:37 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 13:44:23 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id KAA25549 for bluesky@franklin.oit.unc.edu; Thu, 8 Mar 2001 10:41:14 -0800 Date: Thu, 8 Mar 2001 10:41:14 -0800 From: hal@finney.org Message-Id: <200103081841.KAA25549@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: distributed immutable namespace Another question on Adam's proposal: > The idea is to try to create a global namespace > collaboratively maintained by servers which audit each > others behavior and compensate for rogue behavior in > collaboration with client name server protocols. Given that there are multiple servers, how do you deal with database consistency? Specifically, what if two different servers both give out the same name, to different people? Don't you ultimately need a single server, or at least a global tightly-coupled database with locking semantics? How efficient can that be? Even if you design it to do this, how do you resolve disputes when two servers both claim to have followed the rules but both somehow have given out the same name? Now you need audit logs on your global database so you can figure out which one is wrong. I'm afraid that such a system would be limited in the volume of names that it could handle. Maybe it would be OK if we envisioned it being used for a relatively small number of TLDs, and then each of those operates its own name service. Hal From orasis@acm.cs.umn.edu Thu Mar 08 13:50:18 2001 Return-Path: Received: from acm.cs.umn.edu ([128.101.32.12]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 13:49:45 -0500 Received: from go.cs.umn.edu (go.cs.umn.edu [128.101.32.4]) by acm.cs.umn.edu (8.11.2/8.11.0) with ESMTP id f28IrjF74161 for ; Thu, 8 Mar 2001 12:53:45 -0600 (CST) (envelope-from orasis@acm.cs.umn.edu) Received: (from orasis@localhost) by go.cs.umn.edu (8.11.2/8.11.0) id f28Coo938242 for bluesky@franklin.oit.unc.edu; Thu, 8 Mar 2001 12:50:50 GMT (envelope-from orasis) Date: Thu, 8 Mar 2001 12:50:50 +0000 From: Justin Chapweske To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308125050.I33945@go.cs.umn.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from ian@octayne.com on Thu, Mar 08, 2001 at 10:32:50AM -0800 This is very similar to what mindpixel does. Its cool, check it out: http://www.mindpixel.com/ > > The seeds of a solution > I started to think about whether it would be possible to create a system > where all the users, while being tested by the system, were actually > testing each-other too. Each user might be asked to provide an answer to > a question which will demonstrate their intelligence, to provide a > question which will test someone elses intelligence, and to give an > opinion as to the answers given by a number of other people to several > questions. Failure to do any of these things to the best of their > ability may result in their submission not being accepted. > > Remaining issues > Of course, there are still many details to be worked out, how can the > above system be built so as to make abuse impossible? What if a computer > made so many submissions that it saturated the system, and could > validate its own submissions as being correct? > None the less, at least this suggests that a robust think-cash mechanism > is a possibility. As the threat of malicious attacks on the Internet get > more and more serious, people may be forced to adopt a mechanism such as > this. Content-Description: footer > --- > You are currently subscribed to bluesky as: orasis@acm.cs.umn.edu > For list information visit http://www.transarc.ibm.com/~ota/bluesky/ -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From adam@cypherpunks.ai Thu Mar 08 14:12:28 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 14:12:09 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id CA49E5B; Thu, 8 Mar 2001 15:11:59 -0400 (AST) Date: Thu, 8 Mar 2001 15:11:59 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308151159.B87951@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from hal@finney.org on Thu, Mar 08, 2001 at 10:41:14AM -0800 Return-Path: Sender: adam@cypherpunks.ai On Thu, Mar 08, 2001 at 10:41:14AM -0800, hal@finney.org wrote: > Another question on Adam's proposal: > > The idea is to try to create a global namespace > > collaboratively maintained by servers which audit each > > others behavior and compensate for rogue behavior in > > collaboration with client name server protocols. > > Given that there are multiple servers, how do you deal with database > consistency? Specifically, what if two different servers both give > out the same name, to different people? Don't you ultimately need > a single server, or at least a global tightly-coupled database with > locking semantics? How efficient can that be? You need a single server to be responsible for each chunk of namespace as you describe. Other name servers could cache responses, audit other servers, and keep information to be able to undo modifications rogue servers make. > I'm afraid that such a system would be limited in the volume of names > that it could handle. Maybe it would be OK if we envisioned it being > used for a relatively small number of TLDs, and then each of those > operates its own name service. So with hierarchical namespaces like DNS, the risk is that the nameserver with control of a subdomain can censor or be coerced into censoring a name. My suggestion isn't to make it more distributed, but to detect and have a protocol on the client and other servers which allows modifications to be undone. Scalability would be related to DNS, lots of caching helps scalability. Adam From mfreed@MIT.EDU Thu Mar 08 21:02:23 2001 Return-Path: Received: from pacific-carrier-annex.mit.edu ([18.7.21.83]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Thu, 08 Mar 2001 21:02:18 -0500 Received: from central-city-carrier-station.mit.edu (CENTRAL-CITY-CARRIER-STATION.MIT.EDU [18.7.21.75]) by pacific-carrier-annex.mit.edu (8.9.2/8.9.2) with ESMTP id VAA24050 for ; Thu, 8 Mar 2001 21:02:08 -0500 (EST) Received: from melbourne-city-street.MIT.EDU (MELBOURNE-CITY-STREET.MIT.EDU [18.69.0.45]) by central-city-carrier-station.mit.edu (8.9.2/8.9.2) with ESMTP id VAA07821 for ; Thu, 8 Mar 2001 21:02:03 -0500 (EST) Received: from griffen (GRIFFEN.MIT.EDU [18.239.2.119]) by melbourne-city-street.MIT.EDU (8.9.3/8.9.2) with SMTP id VAA06484 for ; Thu, 8 Mar 2001 21:01:20 -0500 (EST) Message-Id: <200103090201.VAA06484@melbourne-city-street.MIT.EDU> X-Sender: mfreed@hesiod X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.2 Date: Thu, 08 Mar 2001 21:00:32 -0500 To: "Global-Scale Distributed Storage Systems" From: Michael J Freedman Subject: Re: distributed immutable namespace: Think Cash In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 10:32 AM 3/8/2001 -0800, you wrote: >This same issue came up on the Freenet mailing list a while back. We >weren't happy with hashcash since it would make your ability to publish >information proportional to the speed of your PC (which is roughly >proportional to the size of your wallet), and we didn't like that at a >philisophical level. This goes back to the question as to the particular need for POWs such as hash cash. Our contention is that for resources which become only temporarily used -- such as bandwidth (Gnutella), CPUs cycles, and caches (Freenet) -- that you don't need a congestion management mechanism *at all times*. Rather, you only need to dynamically change your "publishing" (querying, etc.) requirements based on the current load of your system. That is, as your system gets congested due to temporary load, raise the amount of required payments dynamically. We refer to this as an "accountability slider" in the O'Reilly P2P book. If your resources are cumulatively used -- such as disk space in permanent storage systems (Free Haven) -- then enforcing micropayment requirements when you start getting congested is *already too late.* Thus, for these types of systems, our contention (in the micropayment world) you need 'em all the time. I have some concerns as to the real viability of "Think Cash" in a distributed P2P environment. >task. Thinking of a way to do this is not easy - any such mechanism >would need the following properties: > > 1. Only a human should be able to perform the task or test > 2. A computer should be able to create a test automatically > 3. A computer should be able to judge the results of the test > > >speech recognition. The problem here is not so much creating the tests >(requirement 2), but judging them (requirement 3). There is the further >constraint that each test must have enough possible answers so that a >computer could not simply attempt all of them. > First of all, I think this is actually an incredibly difficult problem which you suggest. Let's describe this in terms of a normal interactive protocol (IP) framework. You are suggesting that your proof system has a prover (P) and verifier (V), such that after the IP, V should only accept (with probability 1-\epsilon, 1-constant, whatever) if P \element {humans}. Getting this (completeness). and soundness (no cheating) with small enough false positives and false negatives is, my opinion, going to be a very hard problem. The issue is compounded that you are working in a distributed peer-to-peer environment. Generally, we would like protocols to be simple, well-understood, and light-weight. This type of system would likely be very complex, hard to formalize, and thus hard to really analyze. You mention the Turing test. Unfortunately, V in that system is a human, not a computer. The ability for computers to perform natural language processing is fairly well along, but it's definately *not* a light-weight operation. But language understanding is much further back, especially if your language domain is fairly large. For example, MIT LCS's Spoken Language Systems group has good working systems for language understanding, but each system only handles a small "problem:" the weather, driving directions, airplane reservations, etc. >The seeds of a solution >I started to think about whether it would be possible to create a system >where all the users, while being tested by the system, were actually >testing each-other too. Each user might be asked to provide an answer to >a question which will demonstrate their intelligence, to provide a >question which will test someone elses intelligence, and to give an >opinion as to the answers given by a number of other people to several >questions. Failure to do any of these things to the best of their >ability may result in their submission not being accepted. This proposals layering many two-way interactions in the "intelligence testing" protocol. It also suggests that users -- who use the system with widely varying educational backgrounds, computer literacy, cultures, comprehension of the technologies involved, etc. -- to "judge" each other's intelligence. And then make operational decisions based on their limited interaction and limited understanding of each other. Make the problems too hard, and your false negatives go way up. Make the problems too each, and I'll write an automated simulator that'll convince most of them. Your false positives go up. Throw some machine learning in there? Then you get away from the "user-interaction" model and you added lots of complexity. When you aren't formalizing the problem -- like hash cash, client puzzles, Naor/Dwork's paying for processing -- you aren't establishing complexity classes, in any sense, for the issues at hand. >Remaining issues >Of course, there are still many details to be worked out, how can the >above system be built so as to make abuse impossible? What if a computer >made so many submissions that it saturated the system, and could >validate its own submissions as being correct? See above. >None the less, at least this suggests that a robust think-cash mechanism >is a possibility. As the threat of malicious attacks on the Internet get >more and more serious, people may be forced to adopt a mechanism such as >this. I fear "robust" is something that hardly applies. Your goal might be an admirable one, but I am very hesitant to think that a "Think Cash" system as you describe is realizable in the near- to medium- future. --mike ----- "Not all those who wander are lost." mfreed@mit.edu From ian@technic.uprizer.com Fri Mar 09 00:11:34 2001 Return-Path: Received: from technic.uprizer.com ([24.19.165.172]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 00:11:14 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f295I4001385 for bluesky@franklin.oit.unc.edu; Thu, 8 Mar 2001 21:18:04 -0800 Date: Thu, 8 Mar 2001 21:18:04 -0800 From: Ian Clarke To: Global-Scale Distributed Storage Systems Subject: Think cash Message-ID: <20010308211804.C1313@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="ZmUaFz6apKcXQszQ" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from mfreed@MIT.EDU on Thu, Mar 08, 2001 at 09:00:32PM -0500 --ZmUaFz6apKcXQszQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 08, 2001 at 09:00:32PM -0500, Michael J Freedman wrote: > I have some concerns as to the real viability of "Think Cash" in a > distributed P2P environment. No interesting problem is ever easy. > First of all, I think this is actually an incredibly difficult problem > which you suggest.=20 Again, no interesting problem is ever easy ;-) > You mention the Turing test. Unfortunately, V in that system is a human, > not a computer. The ability for computers to perform natural language > processing is fairly well along, but it's definitely *not* a light-weight > operation. But language understanding is much further back, especially if > your language domain is fairly large. For example, MIT LCS's Spoken > Language Systems group has good working systems for language understandin= g, > but each system only handles a small "problem:" the weather, driving > directions, airplane reservations, etc. Yes, but if you read on you see that I am actually talking about setting up the system in such a manner that the humans in the system actually test each other, and are incentivised to do a good job. Again, this would be difficult to arrange, but not necessarily impossible. > >The seeds of a solution > >I started to think about whether it would be possible to create a system > >where all the users, while being tested by the system, were actually > >testing each-other too. Each user might be asked to provide an answer to > >a question which will demonstrate their intelligence, to provide a > >question which will test someone else's intelligence, and to give an > >opinion as to the answers given by a number of other people to several > >questions. Failure to do any of these things to the best of their > >ability may result in their submission not being accepted. >=20 > This proposals layering many two-way interactions in the "intelligence > testing" protocol. It also suggests that users -- who use the system with > widely varying educational backgrounds, computer literacy, cultures, > comprehension of the technologies involved, etc. -- to "judge" each other= 's > intelligence. As someone who has spent several years studying Artificial Intelligence (and being largely disappointed by it), one of the most interesting things is that the hardest things for a computer to do, tend to be the easiest things for a human to do, and the converse is also true. I don't think these exchanges would need to test anyone's computer literacy, or educational qualifications. As for cultural differences, that is hardly a problem on the Internet of today! (How many of you are *not* white middle-class males?). Again, I think that you are judging this too harshly, I am not saying that this would be easy, or even possible, but it is fun to think about it, and who knows, maybe something useful will emerge. > Make the problems too hard, and your false negatives go way up. Make the > problems too each, and I'll write an automated simulator that'll convince > most of them. Your false positives go up. Throw some machine learning in > there? Then you get away from the "user-interaction" model and you added > lots of complexity. This is the nice thing about the "reflected turing test" approach, it is a difficult test, but the computer doesn't really have to do the testing (although it does have to arrange things carefully). > When you aren't formalizing the problem -- like hash cash, client puzzles, > Naor/Dwork's paying for processing -- you aren't establishing complexity > classes, in any sense, for the issues at hand. =20 I am not sure what you mean here, but it sounds like you are saying that there is no point in speculation. If that is the case, how do you ever think of anything original or interesting? > >None the less, at least this suggests that a robust think-cash mechanism > >is a possibility. As the threat of malicious attacks on the Internet get > >more and more serious, people may be forced to adopt a mechanism such as > >this.=20 >=20 > I fear "robust" is something that hardly applies. Why not? Just because something is difficult doesn't mean it is impossible. Freenet showed me that. > Your goal might be an admirable one, but I am very hesitant to think that= a > "Think Cash" system as you describe is realizable in the near- to medium- > future. So much pessimism in one so young.... ;-) Ian. --ZmUaFz6apKcXQszQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qGeMQtgxRWSmsqwRAtDnAJ0UzJpPcUt7ImabIkGpmWY+yStEVQCfTkaQ ZkJnG+ro1EAXh8uUg9qct0U= =Iq0p -----END PGP SIGNATURE----- --ZmUaFz6apKcXQszQ-- From mfreed@MIT.EDU Fri Mar 09 01:27:55 2001 Return-Path: Received: from fort-point-station.mit.edu ([18.72.0.53]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 01:27:34 -0500 Received: from central-city-carrier-station.mit.edu (CENTRAL-CITY-CARRIER-STATION.MIT.EDU [18.7.21.75]) by fort-point-station.mit.edu (8.9.2/8.9.2) with ESMTP id BAA15473 for ; Fri, 9 Mar 2001 01:27:27 -0500 (EST) Received: from melbourne-city-street.MIT.EDU (MELBOURNE-CITY-STREET.MIT.EDU [18.69.0.45]) by central-city-carrier-station.mit.edu (8.9.2/8.9.2) with ESMTP id BAA24056 for ; Fri, 9 Mar 2001 01:21:50 -0500 (EST) Received: from griffen (GRIFFEN.MIT.EDU [18.239.2.119]) by melbourne-city-street.MIT.EDU (8.9.3/8.9.2) with SMTP id BAA08597 for ; Fri, 9 Mar 2001 01:21:50 -0500 (EST) Message-Id: <200103090621.BAA08597@melbourne-city-street.MIT.EDU> X-Sender: mfreed@hesiod X-Mailer: QUALCOMM Windows Eudora Pro Version 4.0.2 Date: Fri, 09 Mar 2001 01:21:10 -0500 To: "Global-Scale Distributed Storage Systems" From: Michael J Freedman Subject: Re: Think cash In-Reply-To: References: Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" At 09:18 PM 3/8/2001 -0800, Ian Clarke wrote: >> Make the problems too hard, and your false negatives go way up. Make the >> problems too each, and I'll write an automated simulator that'll convince >> most of them. Your false positives go up. Throw some machine learning in >> there? Then you get away from the "user-interaction" model and you added >> lots of complexity. > >This is the nice thing about the "reflected turing test" approach, it is >a difficult test, but the computer doesn't really have to do the testing >(although it does have to arrange things carefully). > > >> When you aren't formalizing the problem -- like hash cash, client puzzles, >> Naor/Dwork's paying for processing -- you aren't establishing complexity >> classes, in any sense, for the issues at hand. > >I am not sure what you mean here, but it sounds like you are saying that >there is no point in speculation. If that is the case, how do you ever >think of anything original or interesting? What I just meant is that these other solutions are based on computational problems that are accepted as "hard" (assuming that one-way functions exist). When you start requiring user interaction in a different manner, you start added a lot of uncertainty into your models. Various sociological effects come up that may be much less predictable. You're okay if your system users understand what it's trying to do, but if this is made for the average end-user (I think that's a goal of a lot of p2p systems), I'd try to do the following: tcsh% ./ run-bot "c00ld00d" tcsh% [c00ld00d launched...] tcsh% [c00ld00d finds user "Newbie"] c00ld00d: sends you [Newbie] a message: "Hey!!! How r u?!? can u click OK for me and i'll click OK for u? Cool? THX!!!!!" >> >None the less, at least this suggests that a robust think-cash mechanism >> >is a possibility. As the threat of malicious attacks on the Internet get >> >more and more serious, people may be forced to adopt a mechanism such as >> >this. >> >> I fear "robust" is something that hardly applies. > >Why not? Just because something is difficult doesn't mean it is >impossible. Freenet showed me that. I apologize if I came across too strongly. I think what Freenet is trying to achieve is really good, and that a number of really smart people are working towards similar ends. One of the whole reasons for this mailing lists is to discuss our thoughts. The only reason I brought up these points is to try and emphasize some problems in building complex systems. It's hard. I think we are all aware of that. I think we're starting to see some second systems-type effects: we're trying to layer handwaved, complex solutions over handwaved, complex systems. It's always been an endemic problem with system design. =) Hopefully, we can find some good answers to some of these hard problems. Thanks, --mike ----- "Not all those who wander are lost." mfreed@mit.edu From md98-osa@nada.kth.se Fri Mar 09 02:01:28 2001 Return-Path: Received: from mail1.telia.com ([62.20.148.126]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 02:01:21 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14bGwH-0000C5-00 for bluesky@franklin.oit.unc.edu; Fri, 09 Mar 2001 08:03:49 +0100 Date: Fri, 9 Mar 2001 08:03:49 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309080348.A644@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from ian@freenetproject.org on Thu, Mar 08, 2001 at 09:18:04PM -0800 On Thu, Mar 08, 2001 at 09:18:04PM -0800, Ian Clarke wrote: <> > > When you aren't formalizing the problem -- like hash cash, client puzzles, > > Naor/Dwork's paying for processing -- you aren't establishing complexity > > classes, in any sense, for the issues at hand. > > I am not sure what you mean here, but it sounds like you are saying that > there is no point in speculation. If that is the case, how do you ever > think of anything original or interesting? Ian, if people can't criticize or doubt ideas, how will we ever discuss anything original or interesting? -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From alk@pobox.com Fri Mar 09 03:17:56 2001 Return-Path: Received: from poboxer.pobox.com ([199.199.248.17]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 03:17:46 -0500 Received: from spanky.love.edu (spanky [172.30.0.35]) by poboxer.pobox.com (8.9.3/8.9.3) with SMTP id CAA10149 for ; Fri, 9 Mar 2001 02:16:45 -0600 From: Tony Kimball MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15016.37291.224738.364983@spanky.love.edu> Date: Fri, 9 Mar 2001 02:17:47 -0600 (CST) X-Face: \h9Jg:Cuivl4S*UP-)gO.6O=T]]@ncM*tn4zG);)lk#4|lqEx=*talx?.Gk,dMQU2)ptPC17cpBzm(l'M|H8BUF1&]dDCxZ.c~Wy6-j,^V1E(NtX$FpkkdnJixsJHE95JlhO 5\M3jh'YiO7KPCn0~W`Ro44_TB@&JuuqRqgPL'0/{):7rU-%.*@/>q?1&Ed Reply-To: alk@pobox.com To: bluesky@franklin.oit.unc.edu Subject: Think cash References: X-Mailer: VM 6.72 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid It seems not so hard to me. For example, abstractly, the computer can make a picture, and ask a question about it which requires significant understanding, and parse the reply trivially. For example, concretely, the computer might compose many differently colored pictures of fish so that exactly one small fish is within the concavity of the mouth of exactly one, differently colored, larger fish, and ask "which color of fish is about to be eaten by which other color of fish?" The answer can be either a pair of colors or a sentence which is scanned for case and voice, to normalize color order, and color equivalence class word lists are checked to verify the answer. I think the general method is a 2-day hack, tops, with perhaps 30 minutes each to add trivial variations to the dictionary of image relations. Perhaps the real problem here is like the eye tests at the DMV: It costs more think cash to devise the think cash test than it does to build a dictionary of queries and replies! What I find truly difficult is devising think cash tests which do not rely, as this example would have to do in order to be effective, upon "security through obscurity". From ian@technic.uprizer.com Fri Mar 09 03:27:48 2001 Return-Path: Received: from technic.uprizer.com ([24.19.165.172]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 03:27:38 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f298YN702215; Fri, 9 Mar 2001 00:34:23 -0800 Date: Fri, 9 Mar 2001 00:34:23 -0800 From: Ian Clarke To: Oskar Sandberg , Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309003423.G1313@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="rV8arf8D5Dod9UkK" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from md98-osa@nada.kth.se on Fri, Mar 09, 2001 at 08:03:49AM +0100 --rV8arf8D5Dod9UkK Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Mar 09, 2001 at 08:03:49AM +0100, Oskar Sandberg wrote: > On Thu, Mar 08, 2001 at 09:18:04PM -0800, Ian Clarke wrote: > <>=20 > > > When you aren't formalizing the problem -- like hash cash, client puz= zles, > > > Naor/Dwork's paying for processing -- you aren't establishing complex= ity > > > classes, in any sense, for the issues at hand. =20 > >=20 > > I am not sure what you mean here, but it sounds like you are saying that > > there is no point in speculation. If that is the case, how do you ever > > think of anything original or interesting? >=20 > Ian, if people can't criticize or doubt ideas, how will we ever discuss > anything original or interesting? Of course people can criticize ideas, but what I was complaining about (which may or may not be what was intended) was criticising an idea, not on the basis of the idea itself, but on the basis that it is ambitious, or that it is attempting to achieve something where a solution is not immediately obvious. It is the difference between saying "that won't work" and "you will never find a solution". Ian. --rV8arf8D5Dod9UkK Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qJWPQtgxRWSmsqwRAib4AJ9Z1BAm0zfE6lXKeWqOwOpAn8dyUwCeIXHm j7q8+RaHA9KEaPTWxUGIYJ4= =gRnn -----END PGP SIGNATURE----- --rV8arf8D5Dod9UkK-- From adam@cypherpunks.ai Fri Mar 09 03:29:08 2001 Return-Path: Received: from cypherpunks.ai ([209.88.68.47]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 03:28:55 -0500 Received: by cypherpunks.ai (Postfix, from userid 1019) id B0EA457; Fri, 9 Mar 2001 04:28:47 -0400 (AST) Date: Fri, 9 Mar 2001 04:28:47 -0400 From: Adam Back To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309042847.B75666@cypherpunks.ai> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Mailer: Mutt 0.95.6i In-Reply-To: ; from Tony Kimball on Fri, Mar 09, 2001 at 02:17:47AM -0600 Return-Path: Sender: adam@cypherpunks.ai A friend of mine had a go at building such a puzzle to deter automated downloads of activation codes. He built a prototype. It was just a pattern matching problem. It would take a set of numbers draw them in a pixel array and munge it around a bit, scatter random pixels over it. Humans can easily read off the number. It would take a bit of effort to build OCR software that can read through that noise. But I'm pretty sure it could be done if somone were motivated. You see a similar thing where people render their email addresses as a GIF to hide it from spiders. Adam On Fri, Mar 09, 2001 at 02:17:47AM -0600, Tony Kimball wrote: > > It seems not so hard to me. > > For example, abstractly, the computer can make a picture, and ask a > question about it which requires significant understanding, > and parse the reply trivially. From graydon@venge.net Fri Mar 09 08:23:59 2001 Return-Path: Received: from spitfire.velocet.net ([209.167.225.66]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 08:23:48 -0500 Received: from venge.net (H42.C192.tor.velocet.net [216.138.192.42]) by spitfire.velocet.net (Postfix) with SMTP id 957EA19A039 for ; Fri, 9 Mar 2001 08:23:35 -0500 (EST) Received: (qmail 2767 invoked by uid 1000); 9 Mar 2001 13:16:33 -0000 From: graydon@venge.net Date: Fri, 9 Mar 2001 08:16:33 -0500 To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309081632.A2407@venge.net> Reply-To: graydon@pobox.com References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.3.12i In-Reply-To: ; from alk@pobox.com on Fri, Mar 09, 2001 at 02:17:47AM -0600 On Fri, Mar 09, 2001 at 02:17:47AM -0600, Tony Kimball wrote: > It seems not so hard to me. this is precisely the matter which is troubling about these comments. caesar ciphers "seemed" pretty hard too. an awful lot of work has gone into classifying exactly _how_ hard certain problems are, within a framework of more clearly defined mathematics. while it may be true that the human brain has a "leg up" on certain types of problems, we have essentially no theoretical work to tell us _how much_ of a leg up it is, what sort of computational problems the leg up depends on, and why we should think that these computational problems are of the sort where the person with a leg up can consistently "scale" the problem difficulty to adapt to a persistent attacker. moreover, the (brief) bit of cognitive science I got lead me to believe that scaling the computational power of the human brain is not really the sort of thing we know how to do anyways, so even if the leg up is _theoretically_ scalable, I don't see why we should expect a human interface to resist brute forcing, long term. an example of a rediculously crude analysis: even assuming that there is no such thing as pattern matching technology on computers, and they are using memcmp to break your code, and humans have so much of a leg up that they can pick out a _single_ salient bit of data in a full screen truecolor image, the brute force solution is to take 1024 x 768 x (2^24) images and write down what each of them is a picture of. do the math: 1024 768 2 24 ^ 16777216 * 12884901888 * 13194139533312 1500000000 * 1440 * 2160000000000 /> 6.10839793208 so the people's republic of china, looking at 1 image a minute, could completely defeat this system in 6 days. that is a quantitative statement, admittadly bogus, but it illustrates the point: until you ground your problem model in some math, you are _not_ making quantitative statements but rather waving your hands, and the people in the audience with enough experience watching people wave their hands (or enough experience waving our own hands) will be justifiably skeptical. -graydon From dmolnar@hcs.harvard.edu Fri Mar 09 08:50:38 2001 Return-Path: Received: from hcs.harvard.edu ([140.247.73.252]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 08:50:28 -0500 Received: from localhost (dmolnar@localhost) by hcs.harvard.edu (8.9.3/8.9.3) with ESMTP id IAA01350 for ; Fri, 9 Mar 2001 08:50:21 -0500 (EST) Date: Fri, 9 Mar 2001 08:50:21 -0500 (EST) From: dmolnar To: Global-Scale Distributed Storage Systems Subject: Re: Think cash In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Fri, 9 Mar 2001 graydon@venge.net wrote: > so the people's republic of china, looking at 1 image a minute, could > completely defeat this system in 6 days. that is a quantitative Depending on what we're trying to do, slowing down an automated attack may be enough. For password systems, it's not enough. For using think cash as an alternative to proofs of work in order to slow down DoS attacks, maybe it is. Although what always bothers me about these sorts of solutions is that even checking a challenge introduces overhead...so they don't solve the problem so much as defray it. > the audience with enough experience watching people wave their hands (or > enough experience waving our own hands) will be justifiably skeptical. Agreed. Which is why we should start looking for related work and start trying to formalize the idea. For instance, if we assume that a human has a x% chance of recognizing pictures, and a computer is known to have a y% chance, and there's a gap, what can we build from that gap? It seems to me the "right" thing to do would be to abstract out what the problem is and start talking only in terms of the human's advantage. (What notion of "advantage" makes sense here, anyway?) Then we can worry about which problems are actually hard independently of our think cash protocols. Maybe we won't find any provable gaps. Maybe the gaps which we do find will be too small for comfort (I doubt we'll find anything which seems exponentially easier for a human, for instance). I heard recently of a project at CMU which tried to leverage human pattern recognition for passphrases; I'll see if I can find more information about that. This idea of a "reflected Turing test" also sounds interesting. I regret I haven't read Ian's message in enough detail to comment on it fully. Still, it sounds like it might be a good idea if you can assume that lots of people are sitting at their computers in some p2p system and willing to participate. So what kind of model can we build to test whether that's true? -David From pete@retep.yi.org Fri Mar 09 09:29:33 2001 Return-Path: Received: from femail1.rdc1.on.home.com ([24.2.9.88]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 09:29:09 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail3.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010309014606.INAX6398.femail3.rdc1.on.home.com@gw.localdomain> for ; Thu, 8 Mar 2001 17:46:06 -0800 Received: from pete by gw.localdomain with local (Exim 3.22 #1 (Debian)) id 14bBzI-0000Oa-00 for ; Thu, 08 Mar 2001 20:46:36 -0500 Date: Thu, 8 Mar 2001 20:46:36 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: distributed immutable namespace Message-ID: <20010308204636.A1507@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="45Z9DzgjV8m4Oswq" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from adam@cypherspace.org on Thu, Mar 08, 2001 at 12:48:51AM -0400 Sender: Peter Todd --45Z9DzgjV8m4Oswq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 08, 2001 at 12:48:51AM -0400, Adam Back wrote: > Even though ecash to pay for names in a namespace is > several steps removed from content some of which RIAA > may dispute, I doubt a court will feel this any reason > not to rule that the mint be shut down. Perhaps there > is even a precedent of sorts in the MPAA decision > ruling that links to DeCSS source code should be > removed. We have to assume that we are living in a dictatorship where the mear possession of such software is illegal. Looking for loopholes in law is a fruitless venture in the long term. > Hashcash may burn resources, but it appears to be all > that we have in the way of distributed DoS resistance. It's a really great way to get past the centralized authorities needed for standard monetary systems that's for sure, probably the only way. The problem is the inflation issue. The inflation we're going to get is very extreme. Not really a problem in itself but designing algorithms to automaticly update pricing, *without* human intervention, isn't going to be easy. Secondly the inflation will be tough for humans to comprehend. What was worth 100 hashbucks a month ago would now be worth 1000 hashbucks. The interface would have to be constantly making comparisons to the cost of other resources. So instead of saying x will cost y hashbucks instead say x will cost the same amount needed to buy 1meg of bandwidth or something. Hmm... Given a perfect market, which is what we will end up with, the value in hashcash of *any* service is always going to be as low as it possibly can before average costs exceed profit. Sorry if I'm not explaining this well but because *anyone* can mint new money they are only going to bother to offer a service if it's more profitable to do that then just mint new hashcash. This will drive up inflation even more... On the good side the high-inflation is self-limiting. At a certain point it becomes easier to make the hashcash "legitimitly", IE by trading it for bandwidth for instance, then to simply create it because not including Moores law the "cost" to make new hashcash stays constant even if it's value goes down. BTW How excactly is this hashcash supposed to work as a generalized currency? I've got some ideas that I just came up with while trying to figure that out but others almost definetely have better ones. I understand how it can be used to limit access to resources, but what about as a virtual gold item that can be moved and distroyed and is hard to create? > Also hashcash would typically burn resources that would > be revving away idle anyway (modulo the difference in > power saving state some hardware may switch to if the > processor is idle). Unless it got really heated and > people started building expensive custom hardware purely > to escalate the race on desirable namespace. Another > risk for hashcash apart from custom hardware races is > virii -- imagine a virus or worm with a payload which > computed hashcash collisions. That'll happen. It's going to be really nasty when it becomes very profitible to steal computing power. --=20 retep@penguinpowered.com http://retep.tripod.com=20 --45Z9DzgjV8m4Oswq Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qDX8LdZCJKx+dn0RAt5/AJ9NXPkON8Qj4ZSfYcmsL4NszDwFSQCfbT/W YL2/VxZ540s6FqryI13IRwU= =IsLh -----END PGP SIGNATURE----- --45Z9DzgjV8m4Oswq-- From orasis@acm.cs.umn.edu Fri Mar 09 12:29:08 2001 Return-Path: Received: from acm.cs.umn.edu ([128.101.32.12]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 12:28:45 -0500 Received: from sorry.cs.umn.edu (sorry.cs.umn.edu [128.101.32.10]) by acm.cs.umn.edu (8.11.2/8.11.0) with ESMTP id f29HWhF04790 for ; Fri, 9 Mar 2001 11:32:43 -0600 (CST) (envelope-from orasis@acm.cs.umn.edu) Received: (from orasis@localhost) by sorry.cs.umn.edu (8.11.2/8.11.0) id f29HTZg25489 for bluesky@franklin.oit.unc.edu; Fri, 9 Mar 2001 11:29:35 -0600 (CST) (envelope-from orasis) Date: Fri, 9 Mar 2001 11:29:35 -0600 From: Justin Chapweske To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309112934.J23900@sorry.cs.umn.edu> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from dmolnar@hcs.harvard.edu on Fri, Mar 09, 2001 at 08:50:21AM -0500 > > Agreed. Which is why we should start looking for related work and start > trying to formalize the idea. > Again, look at the Mindpixel stuff (http://www.mindpixel.com/). Create an account and start talking to the thing and you will see that the system is pretty resistant to automated conversation from a robot. Whether something like this worked or not, it would be focusing much more energy on artificial intelligence and I think the community as a whole would grow their knowledge rather quickly to learn knew ways to apply this technology. -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From pete@retep.yi.org Fri Mar 09 17:15:46 2001 Return-Path: Received: from femail1.rdc1.on.home.com ([24.2.9.88]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 17:15:22 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail1.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010309221355.LZLI14622.femail1.rdc1.on.home.com@gw.localdomain> for ; Fri, 9 Mar 2001 14:13:55 -0800 Received: from pete by gw.localdomain with local (Exim 3.22 #1 (Debian)) id 14bVAH-0000oO-00 for ; Fri, 09 Mar 2001 17:15:13 -0500 Date: Fri, 9 Mar 2001 17:15:13 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309171513.B3041@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="hHWLQfXTYDoKhP50" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from ian@freenetproject.org on Thu, Mar 08, 2001 at 09:18:04PM -0800 Sender: Peter Todd --hHWLQfXTYDoKhP50 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Mar 08, 2001 at 09:18:04PM -0800, Ian Clarke wrote: > As someone who has spent several years studying Artificial Intelligence > (and being largely disappointed by it), one of the most interesting > things is that the hardest things for a computer to do, tend to be the > easiest things for a human to do, and the converse is also true. I > don't think these exchanges would need to test anyone's computer > literacy, or educational qualifications. As for cultural differences, > that is hardly a problem on the Internet of today! (How many of you are > *not* white middle-class males?). Again, I think that you are judging > this too harshly, I am not saying that this would be easy, or even > possible, but it is fun to think about it, and who knows, maybe > something useful will emerge. What happens when you want to do automated use of resources? Everything that needs thinkcash can't be done by computer alone, quite a disadvantage for many applicaitions IMO. For instance if there was thinkcash on insertion how would you do something like data-based redirects or automated mirroring? --=20 retep@penguinpowered.com http://retep.tripod.com=20 --hHWLQfXTYDoKhP50 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qVXwLdZCJKx+dn0RAnDnAJ9XZG9oDuMpbNf7XdhdYgEHgBM+UgCePRFm Amsjh/wDBudPjfocV5TlT5o= =QVPK -----END PGP SIGNATURE----- --hHWLQfXTYDoKhP50-- From ian@technic.uprizer.com Fri Mar 09 20:47:04 2001 Return-Path: Received: from technic.uprizer.com ([4.18.49.34]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 20:46:47 -0500 Received: (from ian@localhost) by technic.uprizer.com (8.11.0/8.11.0) id f2A1raB04429 for bluesky@franklin.oit.unc.edu; Fri, 9 Mar 2001 17:53:36 -0800 Date: Fri, 9 Mar 2001 17:53:36 -0800 From: Ian Clarke To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309175336.R1510@technic.uprizer.com> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-md5; protocol="application/pgp-signature"; boundary="FWibJpkbnkY6rrXF" Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from retep@penguinpowered.com on Fri, Mar 09, 2001 at 05:15:13PM -0500 --FWibJpkbnkY6rrXF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Fri, Mar 09, 2001 at 05:15:13PM -0500, Peter Todd wrote: > What happens when you want to do automated use of resources? > Everything that needs thinkcash can't be done by computer alone, quite > a disadvantage for many applicaitions IMO. For instance if there was > thinkcash on insertion how would you do something like data-based > redirects or automated mirroring? Well, you need to decide, the things that can be automated can be spammed. One issue is how this could be achieved in a system like Freenet, when you are forwarding a message around, you don't want it to cost thinkcash every time it is transmitted from one node to another. The thinkcash would need to be tied to the CHK of the data being inserted. Ian. --FWibJpkbnkY6rrXF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qYkgQtgxRWSmsqwRApbEAJ918DW3JKqHjteHGfc6v2YTaTeWYwCeOQeo k9GpoPUFOR3sKhHuWlrOzmk= =M+Jk -----END PGP SIGNATURE----- --FWibJpkbnkY6rrXF-- From pete@retep.yi.org Fri Mar 09 22:29:13 2001 Return-Path: Received: from femail1.rdc1.on.home.com ([24.2.9.88]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Fri, 09 Mar 2001 22:28:59 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail1.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010310032733.YLFW14622.femail1.rdc1.on.home.com@gw.localdomain> for ; Fri, 9 Mar 2001 19:27:33 -0800 Received: from pete by gw.localdomain with local (Exim 3.22 #1 (Debian)) id 14ba3m-0000tv-00 for ; Fri, 09 Mar 2001 22:28:50 -0500 Date: Fri, 9 Mar 2001 22:28:50 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010309222850.C3041@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ZmUaFz6apKcXQszQ" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from ian@freenetproject.org on Fri, Mar 09, 2001 at 05:53:36PM -0800 Sender: Peter Todd --ZmUaFz6apKcXQszQ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Fri, Mar 09, 2001 at 05:53:36PM -0800, Ian Clarke wrote: > On Fri, Mar 09, 2001 at 05:15:13PM -0500, Peter Todd wrote: > > What happens when you want to do automated use of resources? > > Everything that needs thinkcash can't be done by computer alone, quite > > a disadvantage for many applicaitions IMO. For instance if there was > > thinkcash on insertion how would you do something like data-based > > redirects or automated mirroring? >=20 > Well, you need to decide, the things that can be automated can be > spammed. One issue is how this could be achieved in a system like Unless we use something like hashcash... > Freenet, when you are forwarding a message around, you don't want it to > cost thinkcash every time it is transmitted from one node to another. > The thinkcash would need to be tied to the CHK of the data being > inserted. Sounds like a easy avenue for a compromise... How can you verify that the thinkcash used for the CHK hasn't been tampered with, say by making it really easy? A hashcash system could simply make a x-bit collision with the CHK itself as verification. --=20 retep@penguinpowered.com http://retep.tripod.com=20 --ZmUaFz6apKcXQszQ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qZ9yLdZCJKx+dn0RAgh3AJ9GZYdWpRoTr2Zy2IL/ettOQUFAJACfV6DN /opG2PF1m/xN7+ZQASui920= =FfL5 -----END PGP SIGNATURE----- --ZmUaFz6apKcXQszQ-- From md98-osa@nada.kth.se Sat Mar 10 08:28:33 2001 Return-Path: Received: from mail1.telia.com ([62.20.149.108]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 10 Mar 2001 08:28:26 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14bjSS-0000E2-00 for bluesky@franklin.oit.unc.edu; Sat, 10 Mar 2001 14:30:56 +0100 Date: Sat, 10 Mar 2001 14:30:56 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010310143056.A696@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from ian@freenetproject.org on Fri, Mar 09, 2001 at 05:53:36PM -0800 On Fri, Mar 09, 2001 at 05:53:36PM -0800, Ian Clarke wrote: <> > Well, you need to decide, the things that can be automated can be > spammed. One issue is how this could be achieved in a system like > Freenet, when you are forwarding a message around, you don't want it to > cost thinkcash every time it is transmitted from one node to another. > The thinkcash would need to be tied to the CHK of the data being > inserted. Humans can spam too. If answering one of these questions takes one second (and it can't take much more for somebody who knows the kind of questions if it's not going to be annoying to normal (stupid) people), than ten people working 16 hours could generate 576,000 such answers. This may be smaller than what a completely automated computer attack could do - but I still wouldn't want 576,000 emails in my box one morning. These sort of systems may shift the financial scale of spamming systems for profit, but a system has to resist even nutty people. And remember - just like the amount of processor power you can have working for you is proportional to your wallet, so is the amount of brain power (my commercial spam operation will outsource the "thinkcash" part to Indonesia). In general, the reason I don't find any type of POW appealing is I don't see the parrallels to the way things happen in meatspace. One of the things that has drawn me to P2P architectures is that they have so many parallels to how things happen in real social systems - and we know such systems have been in the trial and error phase a lot longer than computers. But POW is not how things work in the real world - you don't make a swingset where you have to wind up a spring (uselessly) every couple of swings, just to make sure that some kid doesn't hog the swings all day. It's stupid, and it would take the fun out of swinging. Instead, the other kids gang up, beat up the guy who won't get off, and make him eat sand. That is a sound system for dealing with someone who is abusing resources. I also don't like the wetware bias of the whole thinkcash thing. I would like to see computer agents as welcome as people and Marsians in any system I design, as long as they are good citizens like everybody else. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From pete@retep.yi.org Sat Mar 10 12:35:51 2001 Return-Path: Received: from femail1.rdc1.on.home.com ([24.2.9.88]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 10 Mar 2001 12:35:29 -0500 Received: from gw.localdomain ([24.68.44.193]) by femail1.rdc1.on.home.com (InterMail vM.4.01.03.00 201-229-121) with ESMTP id <20010310173402.UXNW14622.femail1.rdc1.on.home.com@gw.localdomain> for ; Sat, 10 Mar 2001 09:34:02 -0800 Received: from pete by gw.localdomain with local (Exim 3.22 #1 (Debian)) id 14bnGy-0001Ff-00 for ; Sat, 10 Mar 2001 12:35:20 -0500 Date: Sat, 10 Mar 2001 12:35:20 -0500 From: Peter Todd To: Global-Scale Distributed Storage Systems Subject: Re: Think cash Message-ID: <20010310123520.A4665@retep.yi.org> References: Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="ibTvN161/egqYuK8" Content-Disposition: inline User-Agent: Mutt/1.3.15i In-Reply-To: ; from md98-osa@nada.kth.se on Sat, Mar 10, 2001 at 02:30:56PM +0100 Sender: Peter Todd --ibTvN161/egqYuK8 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Sat, Mar 10, 2001 at 02:30:56PM +0100, Oskar Sandberg wrote: > Humans can spam too. If answering one of these questions takes one second > (and it can't take much more for somebody who knows the kind of questions > if it's not going to be annoying to normal (stupid) people), than ten > people working 16 hours could generate 576,000 such answers. This may be > smaller than what a completely automated computer attack could do - but I > still wouldn't want 576,000 emails in my box one morning. 10 people at minimum wage only costs about $64/hour here (Canada) or $1024 to send those 576k messages. This will probably discourage spammers but do little else. And as you say it's easy to outsource this to somewhere like Indonesia, quite possibly making it very profitable. > In general, the reason I don't find any type of POW appealing is I don't > see the parrallels to the way things happen in meatspace. One of the > things that has drawn me to P2P architectures is that they have so many > parallels to how things happen in real social systems - and we know such > systems have been in the trial and error phase a lot longer than > computers. But POW is not how things work in the real world - you don't > make a swingset where you have to wind up a spring (uselessly) every > couple of swings, just to make sure that some kid doesn't hog the swings > all day. It's stupid, and it would take the fun out of swinging. Instead, > the other kids gang up, beat up the guy who won't get off, and make him > eat sand. That is a sound system for dealing with someone who is abusing > resources. Personally I like the idea of hashcash if, and only if, it's structured like a real currency as opposed to simply proof of work. In the real world you pay for resources used. In many cases this should also apply to P2P and other computer systems. Of course getting hashcash workable as a real currency is extremely difficult. I've thought of a scheme that would work (coins are signed by owner and can only be changed (signed to a different owner) by owner) except you need a decentralized "central" database of all the hashcash that's been minted. Unworkable. !@#$ spend-twice problem. :( --=20 retep@penguinpowered.com http://retep.tripod.com --ibTvN161/egqYuK8 Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE6qmXYLdZCJKx+dn0RApfFAJ9SiPS3G6XnNLsxCBB/hmULqPhq5ACghg2Y ktabjDdqeeJlxAsQ+tAEtz0= =K5mX -----END PGP SIGNATURE----- --ibTvN161/egqYuK8-- From admin@madtimes.com Sat Mar 10 17:15:06 2001 Return-Path: Received: from madtimes.com ([208.234.0.42]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 10 Mar 2001 17:14:32 -0500 Received: from classified (mawi83-03.dsl.tds.net [208.170.83.3]) by madtimes.com (8.8.8/8.8.5) with SMTP id RAA28968 for ; Sat, 10 Mar 2001 17:14:23 -0500 Date: Sat, 10 Mar 2001 17:14:23 -0500 Message-ID: <000001c0a9c0$b9279d50$0353aad0@classified> From: "Timm Murray" To: Subject: Re: Naming via SDSI MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Mailer: Microsoft Outlook Express 5.50.4133.2400 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4133.2400 hal@finney.org wrote on 3/6/01 9:08 am: >How would that work in a P2P=20 >model? If we did have=20 >competing name lookup=20 >services (using Wei's term=20 >from the three service=20 >model), would there have to=20 >be some global mechanism=20 >to make sure that no two=20 >services tried to handle the=20 >same addresses? Or would=20 >we accept that address to=20 >name mapping was=20 >inherently ambiguous? Or,=20 >perhaps, would TLDs have to=20 >have unfriendly unforgeable=20 >names? In Freenet Name Service (FNS, see eof.sourceforge.net) there are no TLDs. I suspect people will make psedo-TLDs, but there is no technical reason to do so. Interfacing with other P2P DNS systems is something I have not considered. Thoughts? =0A= Timm Murray=0A= =0A= -----------=0A= Great spirits have allways encountered violent opposition from mediocre = minds=0A= --Albert Einstein From hal@finney.org Sat Mar 10 19:20:38 2001 Return-Path: Received: from finney.org ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sat, 10 Mar 2001 19:20:26 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id QAA03021 for bluesky@franklin.oit.unc.edu; Sat, 10 Mar 2001 16:16:49 -0800 Date: Sat, 10 Mar 2001 16:16:49 -0800 From: hal@finney.org Message-Id: <200103110016.QAA03021@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Naming via SDSI Timm Murray, , writes: > In Freenet Name Service (FNS, see eof.sourceforge.net) there > are no TLDs. I suspect people will make psedo-TLDs, but there > is no technical reason to do so. I see that the specific page describing FNS is http://sourceforge.net/docman/display_doc.php?docid=2162&group_id=15579. FNS is a mechanism for using Freenet to map between names and IP addresses. It is intended as a parallel mechanism to DNS. Unlike DNS, FNS is not hierarchical; names can be any string. FNS mappings are stored in Freenet under Freenet search keys of the form "fns/name/$DATE". Presumably these names are looked up using today's date; for example, "fns/www.microsoft.com/20010310" would be used to look up www.microsoft.com on March 10, 2001. Not only are such mappings first-come, first-served, they are also apparently individually FCFS for every single day. If someone does not create documents for their domain name using a given date string, someone else could step in and in effect take over the domain for that date; is that the intention? Hal From ota@transarc.com Sun Mar 18 18:44:06 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 18 Mar 2001 18:44:00 -0500 Received: from mailhost.transarc.ibm.com (mailhost.transarc.ibm.com [9.38.192.124]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id SAA19038 for ; Sun, 18 Mar 2001 18:38:10 -0500 (EST) Received: from transarc.com (gsine01.us.sine.ibm.com [9.14.6.41]) by mailhost.transarc.ibm.com (8.8.0/8.8.0) with ESMTP id SAA23881; Sun, 18 Mar 2001 18:43:47 -0500 (EST) Message-ID: <3AB54831.69C8FEA1@transarc.com> Date: Sun, 18 Mar 2001 18:43:45 -0500 From: Ted Anderson X-Mailer: Mozilla 4.08 [en] (WinNT; U) MIME-Version: 1.0 To: bluesky@franklin.oit.unc.edu Subject: Root and Branch Naming Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- Subject: Root and Branch Naming It seems that the previous discussion on namespaces, specifically root names, disintegrated without reaching a conclusion. I would like to gently approach the topic from a slightly different angle in hopes of making some progress. There are three aspects of the namimg problem which face different constraints and so probably require diverse solutions. The first level is the root names, this level faces tough non-technical problems. Second is the quasi-static namespaces, below the root, but still high in the naming hierarchy. This level has aggressive performance and reliability requirements. Third, the lower level directories, are not as widely shared, but maybe actively updated and is the control point needing flexible access control and synchronization. For reasons of stability and replication I think the upper parts (but not the top) of the naming hierarchy are best implemented by static directories and described / named by a content-based hash (CHK). Each directory maps names to CHKs, and each CHK unambiguously designates a lower level directory. Because these high levels of the namespace are slowly changing and need to be very widely replicated and cached for performance reasons, it makes sense to use hashes to authenticate and identify them. However, using hashes does work very far down. A change in any leaf node leads to a change in its hash, which changes, its CHK and so requires a change in its parent. This recurrence continues all the way up to the root. Thus a single CHK in the root defines a snapshot of an entire subtree. Changes in the tree are published by releasing new snapshots periodically. Because these trees are static they can be authenticated and cached with high reliability and confidence. At some point, the relaxing performance demands on one hand and the increasing update rate on the other force a change to another mechanism. For lower levels of the namespace that are (relatively) little shared, it makes more sense to have a server / client architecture. In this environment the owner of the directory is a public key which is authorized to update the directory's contents. The transition between the second level hash tree and these online, lower level directories takes the form of a certificate. In the parent, the directory's name is mapped to the CHK of a certificate naming the public key of the directory owner, a list of server locations, and a CHK of a recent snapshot of the directory. This authorizes a server acting on behalf of the owning principal to offer directory services, such as lookup, update, and delegation. Delegation allows nodes with read access to cache a copy of the directory and arrange to be notified of updates. Delegation to a node authenticated as the owner allows the server named in the certificate to offload all request to the delegatee by simply forwarding all requests. The use of delegation would be an optional performance enhancement, which could be implemented by a whole spectrum of mechanism, depending on the load and environment. Delegation could be built upon a high performance algorithm such as used by distributed lock managers found in cluster operating systems, or can be based on a simple scheme that assumes that directories rarely move and the lookup and update load can be handled by a single server. Less optional than delegation is some form of synchronization. Because of the immutability of CHKs, namespace updates are the mechanism for implementing file modifications, namely mapping an existing name to a new contents. Therefore consistency of writes to shared data that may also be cached by readers depends upon some type of locking or synchronization in the namespace. ---***--- I still think that an SPKI (Mark Miller's point that SDSI and SPKI have merged in RFC 2692/3 is well taken), approach is best for handling root names. I would imagine there would be two groups of competing root name authorities, the first group would consist of a handful of conservative organizations. These would only add names that were well defined and widely agreed upon, specifically, within the group of conservative naming authorities. The second group would consist of more liberal naming authorities which would add names relatively quickly and easily. Consequently they might sometime be in conflict with other naming authorities and would sometimes remove names that caused clashes. They would thus provide less stable naming than the first group. Most users would use one authority from the first group to populate their top-level names and possibly use one or more from the second group to fill in behind the primary server. My assumption is that most names users care about would be relatively stable and reliable and that they would tolerate a certain messiness in the management of names around the edges. Since this whole mechanism depends on political and social factors that are undergoing rapid evolution it may be impossible to reliably predict the most successful approach. Flexibility and adaptability will be the most important characteristics. It is worth noting that the top-level of a namespace for a global decentralized file system, is not going to rely heavily on the current host-centric, DNS naming system. Because it is location independent, naming servers is not as relevant as it is in the WWW. Many pathnames will surely be defined by companies and so will have dot-com names near the root. However, other data will be best named by semantic classifications and other taxonomies whose naming authorities are relatively easily established, such as UPC and ISBN. In the long run it isn't clear what kind of global pathnames will be most common or useful. Perhaps understanding this aspect of the naming system will produce more light than heat. In summary, I would suggest that there are three levels of the namespace which require different implementation mechanisms: 1 - root names, depending on political and social issues 2 - hash tree namespaces, providing high performance and stability 3 - online directories, supporting updates and synchronization Ted Anderson -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 7.0.3 for non-commercial use iQCVAwUBOrVHLwGojC9e/wyBAQFbNQQAkZE8zPRi0N2QTJyo8rnjfXSsKx3+hp9Z OEdgA/HZn0AGQGuF3GcDHJfn91mZmIDA7SS37G6dWLJItSgaXoFAmmz4F1jJBfNM wZ0Au9wtMGxMlh98z4DZW9Rb9fk+KRfh1DlPLMKSkS+K/1o2lcT4GwxQDqEbS8Eq PjKzNFnM+MY= =NzhT -----END PGP SIGNATURE----- From ross.anderson@cl.cam.ac.uk Mon Mar 19 04:10:28 2001 X-Mailer: Lyris Web Interface Date: Mon, 19 Mar 2001 04:09:57 -0500 Subject: Re: Root and Branch Naming To: bluesky@ibiblio.org From: "Ross" Ted discusses using hash trees for the higher, more stable levels in the directory structure and more flexible mechanisms lower down. This works; we used something like it for a drug database system. The goal was to provide an electronic equivalent of the drug reference book that doctors carry around with them. We started off with a simple hash tree and authenticated everything by a root hash which changed only when the book was republished every six months. This was far, far more efficient than signing the book - no-one wants to compute a digital signature on 8Mb every time they need to check up on the side effects of aspirin! Then we found a need for flexibility. For example, hospitals annotate particular drugs for cost control reasons: `only Dr Edwards can prescribe this' or `use XXX instead unless authorised by Dr Jones'. Also, additional drugs are added to local databases which don't appear in the national, published one: the classic example is an experimental substance undergoing clinical trials. The trick was to introduce a `flexible link' - basically, instead of hashing the content of the stuff lower down, you hash a public key and sign the stuff lower down with the corresponding private key. This way you can have a trustworthy static book, but with some active pages. There's a bit more you need to do to manage meta-content, as you'd expect. See for the paper we wrote on this; related papers are on the P2P part of my web page, , Ross From hal@finney.org Mon Mar 19 14:27:29 2001 Return-Path: Received: from computer ([207.71.226.132]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 19 Mar 2001 14:27:07 -0500 Received: (from hal@localhost) by finney.org (8.9.3/8.9.3) id LAA01500 for bluesky@franklin.oit.unc.edu; Mon, 19 Mar 2001 11:23:36 -0800 Date: Mon, 19 Mar 2001 11:23:36 -0800 From: hal@finney.org Message-Id: <200103191923.LAA01500@finney.org> To: bluesky@franklin.oit.unc.edu Subject: Re: Root and Branch Naming Ted Anderson, , writes: > There are three aspects of the namimg problem which face different > constraints and so probably require diverse solutions. The first level > is the root names, this level faces tough non-technical problems. > Second is the quasi-static namespaces, below the root, but still high in > the naming hierarchy. This level has aggressive performance and > reliability requirements. Third, the lower level directories, are not > as widely shared, but maybe actively updated and is the control point > needing flexible access control and synchronization. I think you are assuming a relatively deep hierarchical model. Let me first toss out the challenge that we may not need a naming hierarchy at all. Is it really necessary that when I insert a document into the network it has a name like /technical/papers/computerscience/peertopeer/bluesky/problemswithnames.txt? Is someone going to type this in? If not, why not something like Mark Miller's pet-name system [1], 7fa89sdf0a7sf098asf0978asfproblemswithnames.txt, which shows up in your client as "Hal's problemwithnames.txt"? I'd like to see more justification of the need for a hierarchical naming system that fits every single piece of information in the world into a single hierarchy rooted at /. > For reasons of stability and replication I think the upper parts (but > not the top) of the naming hierarchy are best implemented by static > directories and described / named by a content-based hash (CHK). Each > directory maps names to CHKs, and each CHK unambiguously designates a > lower level directory. Because these high levels of the namespace are > slowly changing and need to be very widely replicated and cached for > performance reasons, it makes sense to use hashes to authenticate and > identify them. I see two problems with this. One is the assumption that they are slowly changing. Certainly in the DNS that is not the case. However the DNS is a very shallow hierarchy and you are probably assuming a deep one. Still I suspect that directories near the top are going to be relatively dynamic, because everyone is going to want a short name just like today. Second, it's not clear how a directory informs its parents that an update is needed, in an authenticated way. If /technical/papers/computerscience adds a new subheading, it needs to tell /technical/papers, which then needs to tell /technical, which then needs to tell the root name servers. This notification methodology is outside the scope of your proposal, but it very likely will involve public-key cryptography. This is exactly the technology you propose to deal with the more dynamic directory levels below. Why not just use PKC at all levels? See below for how Freenet proposes to do it. > For lower levels of the namespace that are (relatively) little shared, > it makes more sense to have a server / client architecture. In this > environment the owner of the directory is a public key which is > authorized to update the directory's contents. The transition between > the second level hash tree and these online, lower level directories > takes the form of a certificate. In the parent, the directory's name is > mapped to the CHK of a certificate naming the public key of the > directory owner, a list of server locations, and a CHK of a recent > snapshot of the directory. This authorizes a server acting on behalf of > the owning principal to offer directory services, such as lookup, > update, and delegation. Delegation allows nodes with read access to > cache a copy of the directory and arrange to be notified of updates. > Delegation to a node authenticated as the owner allows the server named > in the certificate to offload all request to the delegatee by simply > forwarding all requests. A few comments on this. First, this is similar to the idea behind the Freenet Signature Verification (or Validation?) Key, SVK. It is essentially a hash of a public key which owns a document. I believe OceanStore also uses this mechanism. Second, this mechanism would be useful for more than just updating directories. What you have described would seem to handle file updates as well. It makes sense to treat directories similarly to ordinary files. Whatever mechanisms we have to deal with changes and updates for directories can be done with files, too. Third, it's not clear that the auxiliary information of server and snapshot are necessary. Freenet and OceanStore use the SVKs as direct indexes into the network. In Freenet they are "first-class" alternatives to CHKs; in OceanStore they are the primary addresses of documents. Given that you have a network that can find data, you don't need to store server addresses. For performance it may be desirable to cache data, but that might be true of leaf-node data as well as directory data. One of the advantages of your explicit-server model is that it allows the key holder more control over updates. Otherwise you do have problems with cache staleness and distribution of updates. Freenet does not have a solution to this yet; OceanStore has a very elaborate mechanism that is supposed to chase down all copies of a document and update it. Last, again it seems that this mechanism would work OK at the top levels as well. The update interval would be larger, perhaps. Then you don't have to redo all the hashes every time something changes three levels down. > I still think that an SPKI (Mark Miller's point that SDSI and SPKI have > merged in RFC 2692/3 is well taken), approach is best for handling root > names. I would imagine there would be two groups of competing root name > authorities, the first group would consist of a handful of conservative > organizations. These would only add names that were well defined and > widely agreed upon, specifically, within the group of conservative > naming authorities. The second group would consist of more liberal > naming authorities which would add names relatively quickly and easily. > Consequently they might sometime be in conflict with other naming > authorities and would sometimes remove names that caused clashes. They > would thus provide less stable naming than the first group. Most users > would use one authority from the first group to populate their top-level > names and possibly use one or more from the second group to fill in > behind the primary server. As I understand this, the root name authorities (RNAs) would not manage the name spaces per se, but would be responsible solely for mapping top level names into top level directories. In the example above, they would map /technical into a CHK which would point at the directory for everything under /technical. Typically the RNAs would support multiple such mappings, and each such mapping would be supported by multiple RNAs. Hopefully there would be substantial agreement among the RNAs about which directory each top level name gets mapped to. This agreement would be essentially 100% among the conservative RNAs, while the liberal RNAs would handle a wider set of names, including those for which there was competition and no consensus yet on which directory was the "real" one for that domain. I think the problem with this model is that it still takes too static a view of the world. While it tries to accommodate change via the liberal RNAs, I think there will be considerable contention even for the names mapped by the conservative ones. Assuming there is some financial advantage in owning a top level name, there will be a constant desire to acquire the most popular names. So I think that we will see continual flux in the providers of these names. What is to prevent the owner of /com or /sex from overcharging for their name registrations, if they have a de facto monopoloy conferred by the conservative RNAs? The only thing that can limit them is fear that others will challenge them by either stealing /com or by starting competing domains. But for these challenges to be credible it must be relatively easy to get new names into widespread use. This implies that even conservative RNAs must be relatively flexible, or they can handle only a small percentage of widely used names. The experience we are getting from the DNS world is that top level names are extremely valuable. The main registry for .com, .org and .net was valued at 21 billion dollars! Given the huge amounts of money to be made here, I don't think it is realistic to assume that the market will be at all stable or orderly. DNS has been limited until now by institutional monopolies. Under competition, with no trademark law to provide property rights in names, it's going to be a wild ride. One thing that's unclear here is how the RNAs themselves are named and found. The impression I have is that they are not part of the globally shared file system (which would produce an awkward recursion problem), but rather they are servers identified as today with DNS names. > In summary, I would suggest that there are three levels of the namespace > which require different implementation mechanisms: > 1 - root names, depending on political and social issues > 2 - hash tree namespaces, providing high performance and stability > 3 - online directories, supporting updates and synchronization I think there are a lot of good ideas here. However I still don't see the root name problem as being solved by letting the market screw in the lightbulb. Institutions like trademarks and courts have evolved worldwide because of exactly the problems that exist when names can be used by anyone. Our admiration for markets should not blind us to the fact that they can't solve all problems. As far as the hashes versus certificates, this may depend on how deep the hierarchy ends up being and how much true stability exists at the highest levels. I would suggest that the system be agnostic about this and allows CHKs and SVKs to be used interchangeably at any directory level. Then technical issues of efficiency and convenience could govern the choice. Hal [1] http://www.erights.org/elib/capability/pnml.html From md98-osa@nada.kth.se Mon Mar 19 16:56:40 2001 Return-Path: Received: from mail1.telia.com ([62.20.149.104]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 19 Mar 2001 16:56:12 -0500 Received: from hobbex by mail1.telia.com with local (Exim 3.16 #1) id 14f7g6-0000NN-00 for bluesky@franklin.oit.unc.edu; Mon, 19 Mar 2001 22:59:02 +0100 Date: Mon, 19 Mar 2001 22:59:01 +0100 From: Oskar Sandberg To: Global-Scale Distributed Storage Systems Subject: Re: Root and Branch Naming Message-ID: <20010319225901.E644@hobbex.localdomain> Mail-Followup-To: Oskar Sandberg , Global-Scale Distributed Storage Systems References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5i In-Reply-To: ; from hal@finney.org on Mon, Mar 19, 2001 at 11:23:36AM -0800 On Mon, Mar 19, 2001 at 11:23:36AM -0800, hal@finney.org wrote: <> > A few comments on this. First, this is similar to the idea behind > the Freenet Signature Verification (or Validation?) Key, SVK. It is > essentially a hash of a public key which owns a document. I believe > OceanStore also uses this mechanism. Actually, I meant it as Signature Verifying Key (the key verifies that the signature is valid) when I made up that acronym, but it seems next to impossible to get people to stick to that (I think somebody passed a law against present participle in TLDs without telling me). Of course, the SVKs have been superseded by the keys that hash in a document name with the key in the final step. It was actually quite silly of me not to realize that that was a good idea from the beginning. Scott called these "SSK" after SVK Subspace Keys (I believe), but I don't like that name so I decided they are called Signed Subspace Keys. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From blanu@uts.cc.utexas.edu Mon Mar 19 20:27:20 2001 Return-Path: Received: from curly.cc.utexas.edu ([128.83.42.1]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 19 Mar 2001 20:27:05 -0500 Received: from localhost (blanu@localhost) by curly.cc.utexas.edu (8.9.3/8.9.3/cc-uts-1.30) with ESMTP id TAA21107 for ; Mon, 19 Mar 2001 19:26:55 -0600 (CST) Date: Mon, 19 Mar 2001 19:26:54 -0600 (CST) From: Brandon X-Sender: blanu@curly.cc.utexas.edu To: Global-Scale Distributed Storage Systems Subject: Re: Root and Branch Naming In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII > One of the advantages of your explicit-server model is that it allows > the key holder more control over updates. Otherwise you do have problems > with cache staleness and distribution of updates. Freenet does not have > a solution to this yet; OceanStore has a very elaborate mechanism that is > supposed to chase down all copies of a document and update it. Freenet does have a solution to this, it just sucks. :-) It's date-based redirects, which allow updating that avoids cache staleness at the expense of having a fixed update interval to which the publisher must adhere. (You must update at every update interval. You cannot update sooner. You cannot miss an update or else the information is hard to find until the next update.) This system works well for periodicals and not very well for arbitrarily updated information such as personal pages. There are some improvements we can do, but this is the basic idea. We'd certainly like something better, more spontaneous. From ota@transarc.com Tue Mar 27 20:41:56 2001 Return-Path: Received: from xfw.transarc.ibm.com ([192.54.226.51]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 27 Mar 2001 20:41:41 -0500 Received: from mailhost.transarc.ibm.com (mailhost.transarc.ibm.com [9.38.192.124]) by xfw.transarc.ibm.com (AIX4.3/UCB 8.7/8.7) with ESMTP id UAA25388 for ; Tue, 27 Mar 2001 20:35:40 -0500 (EST) Received: from transarc.com (gsine06.us.sine.ibm.com [9.14.6.46]) by mailhost.transarc.ibm.com (8.8.0/8.8.0) with ESMTP id UAA25746; Tue, 27 Mar 2001 20:41:18 -0500 (EST) Message-ID: <3AC1413A.F4628EB5@transarc.com> Date: Tue, 27 Mar 2001 20:41:14 -0500 From: Ted Anderson X-Mailer: Mozilla 4.76 [en] (Windows NT 5.0; U) X-Accept-Language: en MIME-Version: 1.0 To: bluesky@franklin.oit.unc.edu Subject: Re: Root and Branch Naming References: <100074@bluesky> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNED MESSAGE----- On 19 Mar 2001 11:23:36 -0800 hal@finney.org wrote: > I think you are assuming a relatively deep hierarchical model. I am indeed assuming a deep hierarchical file system model. My reference point on this is something like AFS, which basically assumes a Unix command line reference model. On the other hand even a stand-alone Windows box has many deep pathnames. If you have 10^20 objects to keep track of, you're going to need a lot of hierarchy. > Let me first toss out the challenge that we may not need a naming > hierarchy at all. Is it really necessary that when I insert a > document into the network it has a name like > /technical/papers/computerscience/peertopeer/bluesky/problemswithnames.txt? > Is someone going to type this in? If not, why not something like Mark > Miller's pet-name system [1], > 7fa89sdf0a7sf098asf0978asfproblemswithnames.txt, which > shows up in your client as "Hal's problemwithnames.txt"? I'd like to > see more justification of the need for a hierarchical naming system > that fits every single piece of information in the world into a single > hierarchy rooted at /. Well my experience is that one very rarely actually types in a long absolute pathname. And when I do, I rely on file name completion to make it easier and catch typos as they arise and not after I've got a 50 character path typed in. Generally, one starts from a working directory or a collection of path abbreviations usually implemented as symlinks from a easy-to-reach starting point. For example, I keep my development sandboxes and many source trees I reference frequently in ~/de, so I can say ~/de/sb1, ~/de/afs3.6 or ~/de/linux-2.4-test7 and quickly get to these far flung parts of the hierarchy. Clearly, this is a little personal namespace and my browser bookmarks work pretty much the same way. So these are pet names of an ad hoc sort. Even the links in a web page are really page-local names for remote pages. However, absolute paths are invaluable for storing in configuration files and scripts and the like so that these things can be shared and the names are the same for everyone. The more collaboration and sharing there is the more important uniform global names are. So I think we need both more and better ways to give objects short easy to remember names as well as long descriptive, uniform, permanent, global names for objects. > > For reasons of stability and replication I think the upper parts > > .. of the naming hierarchy are best implemented by ... [CHKs]. > > I see two problems with this. One is the assumption that they are > slowly changing. Certainly in the DNS that is not the case. However > the DNS is a very shallow hierarchy and you are probably assuming a > deep one. Still I suspect that directories near the top are going to > be relatively dynamic, because everyone is going to want a short name > just like today. This is an issue. First, I think a little more depth would be a good idea. As long as the top level names are kept short, this wouldn't impact conciseness too badly. Second, I assume that while new directory hashes would be produced more or less daily, I don't think that means that everyone needs to get the latest copy every day. If the process is automated I don't see that it would be a big problem to do this once a week or so. Currently, individual nodes don't typically cache these upper-level directories anyway, but contact a nearby name server to request lookups and just cache individual results with a small time-to-live. So the slowly changing assumption mostly applies not to the directories as a whole but to the individual entries. Thus, name mappings are cachable with a simple coherence mechanism. > Second, it's not clear how a directory informs its parents that an > update is needed, in an authenticated way. If > /technical/papers/computerscience adds a new subheading, it needs to > tell /technical/papers, which then needs to tell /technical, which > then needs to tell the root name servers. This notification > methodology is outside the scope of your proposal, but it very likely > will involve public-key cryptography. This is exactly the technology > you propose to deal with the more dynamic directory levels below. Why > not just use PKC at all levels? See below for how Freenet proposes to > do it. This is a good point. I haven't thought very hard about the protocol these guys would use to communicate among themselves. And when I do, I assume some kind of signed update records that are collected by the parent until it is time to generate the next update. This stream of signed updates would work pretty much like a log, but without the usual serialization guarantees that a transaction system provides. One could expose some of this mechanism in the directory format itself which might be more general. In particular, the names in a directory are nearly independent mappings and could be distributed individually or in various groups if that was convenient. For example a very large directory such as the DNS ".com" might actually be divided physically into a 26 sub-directories based on the first letter of the name. Logically, the directory is wide and flat, but physically, it is stored as a two level hierarchy. As Ross pointed out, smaller components allow easier manipulation and verification than huge monolithic objects do. > > ... lower levels [use] a server / client architecture. > Second, this mechanism would be useful for more than just updating > directories. What you have described would seem to handle file > updates as well. It makes sense to treat directories similarly to > ordinary files. Whatever mechanisms we have to deal with changes and > updates for directories can be done with files, too. Yes, sort of. What the owner can do, the no one else is trusted to do is decide which updates to accept. In other words, to exercise access control. In the upper levels this is done by the root name authorities (RNAs) who periodically issue CHKs as detached signatures. While there is public key cryptography (PKC) involved in distributing these CHKs, the signature in the directory, and the protocol for collecting updates from the children, it is only part of the story. The other part is that the RNA uses access control and the application of reputation to the creation of these directories. The same is also true at the lower directory levels and even for files. Someone must exercise access control and that someone has to be the key holder. Another aspect of lower-level directories and some files is they they are accessed like databases. This is to say they are queried and updated (read and written) by multiple parties. Each expects updates to be atomic, consistent, and durable (basically ACID-like) and lookups to be globally serialized with respect to all updates. Some types of files need these properties as well but call the accesses "reads" and "writes". Use a locking protocol to provide isolation in addition to atomicity, and you have ACID semantics. In these cases, a single agent manages object access to provide a coherent view. A single agent is needed to provide both access control and coherence to mutable objects. Most files and some directories, however, are rarely shared or rarely modified and so have little need of database-like semantics. In these cases using CHKs two major benefits. They allow nodes the independence of being able to create globally unique names for objects without contacting a central authority (such as a file server). CHKs also permit easy caching without concern for the source of the object. Ideally, the mechanisms needed for directories and database-like files extend gracefully and efficiently to handle unshared or static files. Nearer the top of the directory hierarchy, very high levels of sharing and relatively static content (in the sense mentioned above) makes database-like mechanisms inappropriate. Instead, periodic snapshots provide approximate, time-bounded consistency, that works well enough most of the time. > Third, it's not clear that the auxiliary information of server and > snapshot are necessary. Freenet and OceanStore use the SVKs as direct > indexes into the network. In Freenet they are "first-class" > alternatives to CHKs; in OceanStore they are the primary addresses of > documents. Given that you have a network that can find data, you > don't need to store server addresses. For performance it may be > desirable to cache data, but that might be true of leaf-node data as > well as directory data. The problem with using the data location network for key hashes is that there is no one-to-one mapping between the key and the data. With CHKs the data is self-certifying. With SVKs one can verify the signature to tell when a datum matches the key, but there is no limit to the amount of data that goes with a single key. This makes the lookup process ambiguous and ill-defined. It makes more sense to put someone in charge of organizing the set of all data signed by a key and provide a way to find that someone. I suggested a list of list of server IP addresses and a snapshot to use if all the servers are down, but there may be better approaches. > One of the advantages of your explicit-server model is that it allows > the key holder more control over updates. Otherwise you do have > problems with cache staleness and distribution of updates. Freenet > does not have a solution to this yet; OceanStore has a very elaborate > mechanism that is supposed to chase down all copies of a document and > update it. As I mentioned above, access control is a crucial aspect of mutable data. Using PK signatures is a decentralized way to implement access control. If the key holder, or his delegate, is online, he can also provide synchronization services (coherence). I am very concerned about the complexity of the OceanStore system. I wish someone who understands it well could comment on the trade-offs made in its design. > > I still think that [SPKI] is best for handling root names. ... > As I understand this, the root name authorities (RNAs) would not > manage the name spaces per se, but would be responsible solely for > mapping top level names into top level directories. In the example > above, they would map /technical into a CHK which would point at the > directory for everything under /technical. Typically the RNAs would > support multiple such mappings, and each such mapping would be > supported by multiple RNAs. Hopefully there would be substantial > agreement among the RNAs about which directory each top level name > gets mapped to. This agreement would be essentially 100% among the > conservative RNAs, while the liberal RNAs would handle a wider set of > names, including those for which there was competition and no > consensus yet on which directory was the "real" one for that domain. This isn't exactly what I was thinking. I was assuming that each user would select a supplier for each top-level name in his private namespace. For example, Verisign might provide me with DNS names, which I would call /dns/com/microsoft and /dns/edu/mit and /dns/org/eff. I might have some other RNA provide other parts of my root namespace, for example, IEEE might provide a compendium of technical research papers available as /research, Project Gutenberg might provide online books categorized by Library of Congress subject headings under /books, RIAA might provide a similar service for /music, SourceForge could provide access to a global source tree under /src, and so forth. In addition, I have my own names for things under /private/mail, /temp, /local/projects, or whatever. Without much trouble, I could allow multiple providers of some names by redirecting misses in the primary to a secondary or tertiary provider. Indeed I could offload this merging behavior to a meta-namespace provider, like today's web meta search engines. I assume that root naming conventions (i.e. /dns, /books, /src) would spring up to ensure that most of the widely shared names would look the same from every node. If someone wanted to organize things differently, however, the only consequence is that global names wouldn't work for him without a translation step. > What is to prevent the owner of /com or /sex from overcharging for > their name registrations, if they have a de facto monopoly conferred > by the conservative RNAs? The only thing that can limit them is fear > that others will challenge them by either stealing /com or by starting > competing domains. But for these challenges to be credible it must be > relatively easy to get new names into widespread use. This implies > that even conservative RNAs must be relatively flexible, or they can > handle only a small percentage of widely used names. The providers of popular names don't get a monopoly quite this easily. If Verisign charges too much for "com/microsoft", then Microsoft can pay GnuDNS instead. As a user, I can choose either Verisign or GnuDNS to provide me /dns, if GnuDNS has more names in it, then I'm more likely to use them than Verisign. As a consequence, Verisign may choose to include "com/microsoft" just to attract users. It might be a bit like competing yellow pages companies. > The experience we are getting from the DNS world is that top level > names are extremely valuable. The main registry for .com, .org and > .net was valued at 21 billion dollars! Given the huge amounts of > money to be made here, I don't think it is realistic to assume that > the market will be at all stable or orderly. DNS has been limited > until now by institutional monopolies. Under competition, with no > trademark law to provide property rights in names, it's going to be a > wild ride. I'm certainly not sure how it would all work out. The dynamics of a global namespace are sure to be complex and unpredicable. But assuming RNAs would need to attract users (subscribers) then they have to compete on the basis of completeness, stability, reliability, etc. They will have to work out some means of agreeing among themselves to provide consistent mappings for most names. > > In summary, I would suggest that there are three levels of the > > namespace which require different implementation mechanisms: > > 1 - root names, depending on political and social issues > > 2 - hash tree namespaces, providing high performance and stability > > 3 - online directories, supporting updates and synchronization I'd expand on this list a bit: 1 - root names, defined by convention; social, political, and economic issues. 2 - hash tree namespaces, providing high performance and stability with weak, time-based consistency. 3 - online directories and database-like files, access control and strong consistency. 4 - private and static data, CHKs provide independence and sharing. > I think there are a lot of good ideas here. However I still don't see > the root name problem as being solved by letting the market screw in > the lightbulb. Institutions like trademarks and courts have evolved > worldwide because of exactly the problems that exist when names can be > used by anyone. Our admiration for markets should not blind us to the > fact that they can't solve all problems. I agree. It may be that for competing RNAs to provide the level of service their customers demand will require a level of cooperation that won't look much like competition in usual commercial sense. The key is giving users the ability to vote with their feet. Ted -----BEGIN PGP SIGNATURE----- Version: PGPfreeware 7.0.3 for non-commercial use iQCVAwUBOsDF+QGojC9e/wyBAQEGzAQA1DxcFjZ1U5CZqFpVH3nUUrq6U1P2WAkZ NIoh4sxCyWYWzwtGXNPMBnejQvEA/amWal2xZGPReEtoWcFUAPiArKexJmX+alWo Gm9tzV/uE5AGhP3k1WKEC4tFgtruhIoNUjt60dgveF103iAiGH6JfSRYRs0yIwqi yFPHTgvYa3s= =ezHN -----END PGP SIGNATURE----- From ian@hawk.freenetproject.org Tue Apr 17 14:05:25 2001 Return-Path: Received: from hawk.freenetproject.org ([4.18.42.11]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Tue, 17 Apr 2001 14:04:47 -0400 Received: by hawk.freenetproject.org (Postfix, from userid 500) id A17FE5801F; Tue, 17 Apr 2001 11:10:10 -0700 (PDT) Date: Tue, 17 Apr 2001 11:10:10 -0700 To: bluesky@franklin.oit.unc.edu Subject: Think Cash - an implementation! Message-ID: <20010417111010.J11279@freenetproject.org> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="fKov5AqTsvseSZ0Z" Content-Disposition: inline User-Agent: Mutt/1.2.5i From: ian@hawk.freenetproject.org (Ian Clarke) --fKov5AqTsvseSZ0Z Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable To my amazement, someone has actually tried to implement some of my ideas around "Think Cash" (see http://www.half-empty.org/ideas/a7/dc/9b/f4/21/default_index.html). You can see the results on Freenet at: MSK@CHK@8GfuCtnk0S~aq2VrJALgi3V2oYYLAwE,kEMmdndjbqUfU8U06kJONQ//dl.html Or if you don't have Freenet, but do have WWW access, try: http://mu2.espnow.com:8081/MSK%40CHK%408GfuCtnk0S~aq2VrJALgi3V2oYYLAwE,kEMm= dndjbqUfU8U06kJONQ//dl.html=20 It is very cool indeed. Ian. --fKov5AqTsvseSZ0Z Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.0.4 (GNU/Linux) Comment: For info see http://www.gnupg.org iD8DBQE63IcCQtgxRWSmsqwRAqwMAJ0dx2L9kf0ExBzPbnXeg5ath9tutACeL0bc nvBHhHfclS62FaULibCdFhU= =peRF -----END PGP SIGNATURE----- --fKov5AqTsvseSZ0Z-- From dnm@pobox.com Wed Jul 11 12:34:22 2001 Return-Path: Received: from mail.opencola.com ([216.94.182.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 11 Jul 2001 12:34:19 -0400 Received: from [10.7.1.129] (block.it.ca [216.126.86.6] (may be forged)) by mail.opencola.com (8.11.0/8.11.0) with ESMTP id f6BFa2825080; Wed, 11 Jul 2001 11:36:07 -0400 Mime-Version: 1.0 X-Sender: dnm@mail.vex.net Message-Id: Date: Wed, 11 Jul 2001 12:33:38 -0400 To: Ted Anderson From: Dan Moniz Subject: OpenCol Swarmcast Cc: bluesky@franklin.oit.unc.edu, Justin Chapweske , Ry4an Brase Content-Type: text/plain; charset="us-ascii" ; format="flowed" Hi Ted, I just came across your web page about the bluesky mailing list and noticed OpenCola Swarmcast mentioned. One correction: Swarmcast isn't commercial, it's open source . Bias disclosure: (my work .sig tells all...) -- Dan Moniz [http://www.opencola.com/] Research Scientist and Chief Security Architect OpenCola, 160 Bloor St E Suite 160, Toronto ON M4W 1B9, Canada V: +1 416 934 5520 | F: +1 416 966 9596 | M: +1 416 896 3266 Thanks! -- Dan Moniz [http://www.pobox.com/~dnm/] From dnm@pobox.com Wed Jul 11 14:38:58 2001 Return-Path: Received: from mail.opencola.com ([216.94.182.2]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Wed, 11 Jul 2001 14:38:55 -0400 Received: from [10.7.1.129] (block.it.ca [216.126.86.6] (may be forged)) by mail.opencola.com (8.11.0/8.11.0) with ESMTP id f6BHej827062; Wed, 11 Jul 2001 13:40:45 -0400 Mime-Version: 1.0 X-Sender: dnm@mail.vex.net Message-Id: Date: Wed, 11 Jul 2001 14:38:24 -0400 To: decentralization@yahoogroups.com, p2p-hackers@zgp.org, bluesky@franklin.oit.unc.edu From: Dan Moniz Subject: dLoo and BlueBox Content-Type: text/plain; charset="us-ascii" ; format="flowed" From the language-dev mailing list: >Date: Wed, 11 Jul 2001 10:16:48 -0600 >From: Nathan Torkington >To: perl6-language@perl.org >Cc: language-dev@netthink.co.uk >Subject: dLoo releases peer-to-peer programming language > > >From newsforge: > >nile writes, "Today, dLoo released the complete architecture of an >extensible peer-to-peer programming language. Unlike traditional languages, >this language is defined on the Internet. Its syntax and semantics can be >extended by posting additional pieces of the language. As developers add to >the language it scales in richness and functionality. Programs run in a >software browser called BlueBox which dynamically downloads and assembles >the parts of the language as needed. For more information and access to the >source visit http://www.dloo.org. BlueBox is a community driven project >released under the GPL." >http://www.dloo.org/ -- Dan Moniz [http://www.pobox.com/~dnm/] From josh@mercuryfs.net Sun Jul 29 20:54:47 2001 X-Mailer: Lyris Web Interface Date: Sun, 29 Jul 2001 20:54:44 -0400 Subject: i've got a design you guys would be interested in To: bluesky@franklin.oit.unc.edu From: "Josh" Hello, I've designed a permanent caching location independent global file system. As such, it appears to be the only design to solve the duplication problem, which is inherent with Internet traffic. I'm just now starting to read thru the past emails, and look forward to speaking with some of you. As techies, I recommend you go straight to the presentation, which shows the essence of permanent caching and location independence. Its at www.mercuryfs.net/presentations/intro_to_mfs.htm - josh www.mercuryfs.net From wesley@felter.org Sun Jul 29 22:40:51 2001 Return-Path: Received: from valentine.felter.org ([64.39.30.19]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Sun, 29 Jul 2001 22:40:48 -0400 Received: from localhost (wesley@localhost) by valentine.felter.org (8.9.3/8.9.3) with ESMTP id PAA22133 for ; Sun, 29 Jul 2001 15:45:47 -0500 Date: Sun, 29 Jul 2001 15:45:47 -0500 (CDT) From: Wesley Felter To: Global-Scale Distributed Storage Systems Subject: Re: i've got a design you guys would be interested in In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII On Sun, 29 Jul 2001, Josh wrote: > Hello, I've designed a permanent caching location independent global file > system. As such, it appears to be the only design to solve the duplication > problem, which is inherent with Internet traffic. How is your system different from OceanStore? > As techies, I recommend you go straight to the presentation, which shows > the essence of permanent caching and location independence. Its at > www.mercuryfs.net/presentations/intro_to_mfs.htm This appears to be in a weird format, so I couldn't view it. Wesley Felter - wesley@felter.org - http://felter.org/wesley/ From josh@mercuryfs.net Mon Jul 30 00:57:31 2001 Return-Path: Received: from server2600.net ([209.239.42.83]) by franklin.oit.unc.edu with SMTP (Lyris List Manager SOLARIS/SPARC version 4.0); Mon, 30 Jul 2001 00:57:28 -0400 Received: from koi (lsanca1-ar2-081-053.lsanca1.dsl.gtei.net [4.35.81.53]) by server2600.net (8.10.2/8.10.2) with SMTP id f6U4v7713109 for ; Mon, 30 Jul 2001 00:57:08 -0400 From: "Josh" To: "'Global-Scale Distributed Storage Systems'" Subject: RE: i've got a design you guys would be interested in Date: Sun, 29 Jul 2001 21:49:49 -0700 Message-ID: <000001c118b3$110753b0$0100007f@koi> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook CWS, Build 9.0.2416 (9.0.2911.0) In-Reply-To: Importance: Normal X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 To be honest, I've never heard of ocean store. Thanks, I will look it up immediately. I found out about freenet via an attorney. I've been so focused on the design that I have