Skip to Content.
Sympa Menu

bluesky - Re: Scalability

bluesky AT lists.ibiblio.org

Subject: Global-Scale Distributed Storage Systems

List archive

Chronological Thread  
  • From: hal AT finney.org
  • To: bluesky AT franklin.oit.unc.edu
  • Subject: Re: Scalability
  • Date: Mon, 26 Feb 2001 23:00:15 -0800


Adam writes:
> - a system to find the URLs you want
> - a system to locate a copy of and download the URL content
>
> Steve Schear and I were discussing the first one and another
> approach is to just let a search engine do it for you. Either
> by exposing the URLs to the real web through gateways which
> make it appear (to search engines only perhaps) as static content.

Web spidering does work to some extent, but not really that well.
A substantial fraction of web pages aren't indexed. It should be possible
to do better if the network were designed for it from the beginning.

> Locating a copy of a target URL to download is a different
> problem. Graydon's log map seems like a reasaonable approach.

What is this? Is it similar to the OceanStore method of mapping
a tree onto a network? I'm not familiar with this data structure.

Hal




Archive powered by MHonArc 2.6.24.

Top of Page