Skip to Content.
Sympa Menu

internetworkers - Re: [internetworkers] free site crawler app?

internetworkers AT lists.ibiblio.org

Subject: Internetworkers: http://www.ibiblio.org/internetworkers/

List archive

Chronological Thread  
  • From: "Scott Lundgren" <internetworkers AT capitalfellow.com>
  • To: scott AT scottboggsdesign.com, "Internetworkers: http://www.ibiblio.org/internetworkers/" <internetworkers AT lists.ibiblio.org>
  • Subject: Re: [internetworkers] free site crawler app?
  • Date: Wed, 31 May 2006 14:49:32 -0400 (EDT)

> I'm working on redesigning a very large intranet site and need to create a
> site map of the whole behemoth. I am wishing that some application exists
> that would crawl the existing site and output a site map, chart, or just a
> list of the site tree.
>

With an internal site you won't be submitting the site to Google but a
quick XSLT will get you what you want.

http://goog-sitemapgen.sourceforge.net/

The sitemap_gen.py script analyzes your web server and generates one or
more Sitemap files. These files are XML listings of content you make
available on your web server. The files can be directly submitted to
search engines as hints for the search engine web crawlers as they index
your web site.






Archive powered by MHonArc 2.6.24.

Top of Page