Skip to Content.
Sympa Menu

bluesky - Re: the three-services model

bluesky AT lists.ibiblio.org

Subject: Global-Scale Distributed Storage Systems

List archive

Chronological Thread  
  • From: hal AT finney.org
  • To: bluesky AT franklin.oit.unc.edu
  • Subject: Re: the three-services model
  • Date: Sat, 17 Feb 2001 09:25:24 -0800


Wei writes:
> I have been assuming a storage system design that decomposes into three
> services:
>
> 1. name lookup service: maping user-friendly names/URLs to content-hash
> based ids
> 2. data location service: maping content-hash based ids to storage servers
> (network addresses and data transport protocols) that can deliver the data
> 3. data transport service: actually obtaining content from a storage
> server given its address and a content-hash based id

Not all services seem to fit this model. Mojonation does not have user
friendly names AFAIK. The end result of their publication process is a
"dinode" consisting of 8 SHA-1 hashes concatenated. One example is:
http://localhost:4004/Dinode/eNpl0stuglAQBuBXIezRUzkCJogBFOQmKFYsGwNyFO
VmQUR8-mrF9MLyn8n3ZxbDjq5JjF1QXhyydIi_dQA-4tjxIc0C9DMGncF9gRWJdzoEQ3zh9
I8CszPNCb28gSxMPnwH0sQiTysci1G6P4f3KgpSDM6xdujlCPPjzH_QIkwMay_BbbYKFUOn
qBUpvlfmji4NHEvLZIjTePc_qnZSTUs6kTOEbMtH5Lo5n9pz6F32DaLaKCXsYB1pNC-TSlk
CBiy3B30r-ycSNKjfRusprzPL8xUyO3GRXdcilGI1VJPN6XUebKN5ZFaWtrbUebS3bqqxEU
4C_5kfqRnfILKNavoqlxpZbatc7glFbToXy9RdxNuv83ptpMWRPwahOOgrK7UoAqcWtdtGZ
oLVvEFvbeSDcDp2XEXyILFR6rMr9RN14gtpNGkQeKBldvbib1lwDNv9HZ-FxQyhAAUcZLt_
Mtt9vgv3BbSVu-I/. (I've broken it up into multiple lines.)

Likewise I don't think Napster or Gnutella have a URL or name for
persistent addressing of data.

Instead, these systems are search based. When you want to find data, you
do a search providing various keywords of substrings. Data is published
on the network with associated meta-data that can include title, author,
etc. Your searches use this information to find data that you might want.
This is presented to you and you then choose which files to download.

Once the search is done you might or might not need to go through steps
2 and 3 above. I think in MojoNation you do go through two more steps.
The search gives you the dinode, but that still has to be converted into
location information which may involve more lookups (I'm not sure about
this part). In Napster and Gnutella I believe the search returns IP
addresses which are sufficient locators already.

> This results in two levels of indirection. Some current systems seem to
> have only one level, either
>
> a. combining (1) and (2), so that a name is mapped directly to a storage
> server, or
> b. combining (2) and (3), so that storage servers are addressed by
> content-hash base ids.
>
> Having an extra level of indirection allows problems with both (a) and (b)
> to be avoided. The problems with (a) are that the storage server has to be
> trusted, and that there is waste if the same data object goes by several
> names in different namespaces and is stored on multiple servers.

I think the trust issue with (a) is present in step 1 in the three
step model. This is a hard problem which has been much discussed on the
Freenet list. How do you trust the name to address mapping? Do you
need to introduce cryptography, a trust model, a PKI? Such systems
have never been successful, although it might be argued that P2P is the
"killer ap" for end-user PKI.

> The
> problems with (b) are that if a data object changes, it would have to be
> relocated to a different server, and that servers have little flexibility
> in choosing what to store.

These problems exist in Freenet, but they don't seem that severe.
Data objects in that system are considered fluid and don't have a
permanent home, flowing through the network. Any system which will avoid
the slashdot effect (what Adam calls flash crowds) needs to be able to
migrate and spread data according to demand. As for servers not being
able to choose what they store, in Freenet it is a feature not a bug,
as it is considered to add to the censorship resistance of the network.

Hal




Archive powered by MHonArc 2.6.24.

Top of Page