ch-scene AT lists.ibiblio.org
Subject: RTP-area local music and culture
List archive
- From: grady <grady AT ibiblio.org>
- To: RTP-area local music and culture <ch-scene AT lists.ibiblio.org>
- Subject: Re: The Future Of Music
- Date: Wed, 06 Oct 2004 13:03:48 -0400
bendy(a)lostpla[dot]net wrote:
grady wrote:
> Or, my personal current fave meme of
the week: using a wget script to scrape mp3s from multiple mp3 blogs and then store them in a folder hierarchy that indicates which mp3 blog they came from. You get 100s of new mp3s to listen to, and if you notice something that interests you, you can figure out where it came from & go looking for more . . .
Do example scripts exist or this memevaporware?
Aw, c'mon: http://www.google.com/search?q=wget+mp3+blog
Here's a sample of some of the methods I used for some of the blogs I scraped. And yes, I feel 12% guilty for not taking the time to read most of 'em before slurping. Then again, most of them just read like blogs. I will say that this is primarily an experiment to see if any of the bloggers are consistently on-point enough for me to start checking them on a regular basis. I don't really want to spend my days slogging through 400 new mp3s/week if 390 of them are bad eurodisco.
(these are all for the win32 port of wget, so minor syntactical differences may or may not exist if you're on a different platform)
(note that in each case, I'm calling the URLs of the blogs from textfiles that are referenced on the cmd-line - you'll have to make your own)
(note also that those cmd-lines are being wrapped by the mailer & you'll need to ensure that they wind up all on one line in your shell before you execute them):
domains where you'll want to go offsite (to random other sites) but only 1 level:
http://www.fatplanet.com.au/mp3_archive.html
http://www.livejournal.com/community/talkiewalkie/
http://fingertipsmusic.blogspot.com/
http://www.3hive.com/
wget -np -t1 --limit-rate=800000 -w1 -nc -x -U "Mozilla/5.001 (windows; U; NT4.0; en-us) Gecko/25250101" -A .mp3,.ogg --follow-ftp -r -l1 -erobots=off -H -i extsites.txt
domains where everything is in a single browsable directory & this is it, man:
http://www.o-dub.com/sounds/soulsides/
http://www.moistworks.com/mp3s/
http://www.bellybongo.se/mp3/
http://tofuhut.racknine.net/
http://members.aol.com/beastmystical/music_files/
http://saturn5.com/~moebius/muzik/
wget -t1 -np --limit-rate=800000 -w1 -nc -x -U "Mozilla/5.001 (windows; U; NT4.0; en-us) Gecko/25250101" -A .mp3,.ogg -r -l4 -erobots=off -i ezsites.txt
domains where everything is onsite but scattered all over:
http://www.christopherporter.com/
http://www.scissorkick.com/
http://www.bubblegum-machine.com/culchah.html
http://www.londonlee.com/blog.html
wget -t1 -np --limit-rate=800000 -w1 -nc -x -U "Mozilla/5.001 (windows; U; NT4.0; en-us) Gecko/25250101" -A .mp3,.ogg --follow-ftp -r -l4 -erobots=off -i intsites.txt
domains where everything is offsite but on a predictable other server:
http://www.pop77.com/blog/
http://www.tangmonkey.com/blogs/music/
http://teachingtheindiekidstodanceagain.blogspot.com/
http://music.for-robots.com/
http://copycommaright.blogspot.com/
http://the-big-ticket.blogspot.com/
http://vinyljourney.blogspot.com/
offsite server list for the above:
http://s93123386.onlinehome.us
http://homepage.mac.com/sceldred/.Public/Steve/
http://turbine.slackworks.com/robots/blair/
http://homepage.mac.com/mr.gilbert/.Public/
http://home.comcast.net/~lobstar72/
http://projectmersh.com/
wget -t1 -np --limit-rate=800000 -w1 -nc -x -U "Mozilla/5.001 (windows; U; NT4.0; en-us) Gecko/25250101" -A .mp3,.ogg --follow-ftp -r -l1 -erobots=off -H -D s93123386.onlinehome.us,homepage.mac.com,turbine.slackworks.com,home.comcast.net,projectmersh.com -i knownsites.txt
I don't really want/need to provide a wget tutorial at the moment, so if you're wondering what all those switches do, do a wget --help & find out. Note that you can remove the rate-limit & the wait between downloads & etc if you want to be a bastard. You can also remove the browser-spoofing; I will freely admit that I only did that to avoid getting caught by any anti-bot/slurp snares that might have been out there.
Yeah, I know, I suck.
xo
Ross
-
If the Indy award posed with a gun, would you vote anyway?,
Thom Wiley, 10/05/2004
- Re: If the Indy award posed with a gun, would you vote anyway?, Doctor Oakroot, 10/05/2004
- Re: If the Indy award posed with a gun, would you vote anyway?, mike barker, 10/05/2004
- <Possible follow-up(s)>
-
RE: If the Indy award posed with a gun, would you vote anyway?,
Fiona Morgan, 10/05/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
grady, 10/05/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
bendy(a)lostpla[dot]net, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
grady, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
bendy(a)lostpla[dot]net, 10/06/2004
-
The Future Of Music (was: Re: If the Indy award posed with a gun, would you vote anyway?),
grady, 10/06/2004
- Re: The Future Of Music, bendy(a)lostpla[dot]net, 10/06/2004
- Re: The Future Of Music, grady, 10/06/2004
- Re: The Future Of Music, Chris Calloway, 10/07/2004
- Re: The Future Of Music, grady, 10/07/2004
- Re: The Future Of Music, grady, 10/07/2004
- Re: The Future Of Music, Christiane Voisin, 10/07/2004
- Re: The Future Of Music, Chris Calloway, 10/07/2004
- Re: The Future Of Music, My friend Phil said,, 10/08/2004
-
The Future Of Music (was: Re: If the Indy award posed with a gun, would you vote anyway?),
grady, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
bendy(a)lostpla[dot]net, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
grady, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
bendy(a)lostpla[dot]net, 10/06/2004
-
Re: If the Indy award posed with a gun, would you vote anyway?,
grady, 10/05/2004
- RE: If the Indy award posed with a gun, would you vote anyway?, Doctor Oakroot, 10/05/2004
Archive powered by MHonArc 2.6.24.