Skip to Content.
Sympa Menu

bluesky - Re: market-based data location (was Scalability)

bluesky AT lists.ibiblio.org

Subject: Global-Scale Distributed Storage Systems

List archive

Chronological Thread  
  • From: Oskar Sandberg <md98-osa AT nada.kth.se>
  • To: Global-Scale Distributed Storage Systems <bluesky AT franklin.oit.unc.edu>
  • Subject: Re: market-based data location (was Scalability)
  • Date: Wed, 28 Feb 2001 03:02:23 +0100


On Tue, Feb 27, 2001 at 09:20:16AM -0800, hal AT finney.org wrote:
> Wei writes:
<>
> There is a file describing their plans for distributed metatracking at
> http://cvs.sourceforge.net/cgi-bin/cvsweb.cgi/~checkout~/evil/hackerdocs/distributed_metatracking.html?cvsroot=mojonation
>
> However I am somewhat worried by:
>
> This idea is simply "every broker a metatracker". I think we can use
> the current metatracker protocols and the current source code to deploy
> really good distributed metatracking.
>
> I'm not sure it really moves us forward much if every node stores data,
> every node runs trackers to index data, and now every node will run a
> metatracker to index the trackers. It's starting to sound like "turtles
> all the way down".

I approached the Mojonation people regarding my concern that the whole
content tracking system was not really decentralized at all at the Peer to
Peer conference. Even if this system works in other respects, with n
documents on the network, you need to satisfy:

<number of docs per tracker> * <number of trackers that users need to
search> = n

Since both these numbers need to be kept down, the best result would be
sqrt(n) for both, but sqrt(n) not really good enough for me to consider
the network decentralized (in the sense of many documents not depending on
few nodes) or scalable (in the sense of each user request not taking
unreasonable work).

It turned out however, that Mojonation's basic architecture isn't scalable
anyways. The content-trackers in MN do not map metadata onto a physical
address, they map metadata onto UIDs (the SHA1 hash for each part) of the
data. Apparently, to then find the data, MN simply expects each node to
keep a list of the IDs of every other node in the network, and then
compares the least order bytes of the node IDs with the data UID to find
the node where to put or look for a piece of data.

With this is mind, complaining about the scalability of the other aspects
of their network seems a little silly.


--
'DeCSS would be fine. Where is it?'
'Here,' Montag touched his head.
'Ah,' Granger smiled and nodded.

Oskar Sandberg
md98-osa AT nada.kth.se




Archive powered by MHonArc 2.6.24.

Top of Page