Hello all,
I like a good rant. I also like the meta rant, or rant about rant. But back to the initial question…….
Paul got all of the legal stuff spot on - the NLNZ required a legislative change in order to scale up our web harvest activities - and I thank him for posting the relevant
links.
Danyl, to answer one of your questions - I do it. Or perhaps more specifically several people within the Digital Collections Strategy team at National Library of NZ do it.
I'm happy to answer any questions you have about it. Feel free to email me if you have any. Interestingly, we use the Web Curator Tool, of which the wayback machine is just one component used in the indexing process, which NLNZ developed in conjunction with
the British Library. You can also download WCT from sourceforge
http://webcurator.sourceforge.net/
Some stats:
We have selected about 14000 web sites since 1999.
This is augmented by 2 whole of domain harvests in 2008 (when it was 4 TB) and 2010 (10TB). The next whole of nz domain harvest is planned to take place in 2013, and there'll
be some more communications about this shortly. We predict it will be approximately 15TB.
We're yet to provide access to our whole of domain harvests. There's a bunch of additional legal and accessibility issues we're working through.
But despite all this work, we'll never get the entire web memory of NZ. Like all collecting activity, it has to function within the truisms of:
1. Everything potentially has value
2. We can't keep everything
3. We can't predict future use
So we've got to run a programme of collecting that uses contemporary archival methodology. It's good fun.
Also, here's an early article on the NZ web archive activity
http://www.dlib.org/dlib/may08/paynter/05paynter.html
I hope this helps,
Mark
Mark Crookston
Digital Collection Strategy Leader
The Alexander Turnbull Library
National Library of New Zealand Te Puna Matauranga o Aotearoa
The Department of Internal Affairs Te Tari Taiwhenua
DDI: +64 4 4704492
http://www.natlib.govt.nz
From: cc-nz-bounces AT lists.ibiblio.org [mailto:cc-nz-bounces AT lists.ibiblio.org]
On Behalf Of Matt McGregor
Sent: Saturday, 1 December 2012 11:45 a.m.
To: Creative Commons Aotearoa New Zealand discussion
Subject: Re: [Cc-nz] Archiving the web for posterity
Hi Leigh,
That's a somewhat fair response, though with respect I think you're conflating a few differing issues. My brief message addressed what you call belittling. I know this is the Internet, but I truly see no reason why anyone needs to belittle anyone--on this
list or anywhere.
On your other points, I (mostly) agree (and I'm not sure why you would think otherwise). Heated discussion should be encouraged. We're debating issues we hold in common--culture in the broadest sense of the term--and the debate could hardly be too lively.
Perhaps the word 'cordial' tipped you off. To be clear, I was making any claims that the general tone of the list--which I think is fantastic. I was referring to a very specific tone. I don't think we should conflate the two.
On the question of silence: You might be right that I silenced the thread, though perhaps we might wait longer than the 12 hours from 10pm Friday to 10am Saturday to make that call?
As I hope this response shows, I do enjoy a heated exchange, and would always encourage others to join in!
Best wishes,
Matt
On 1 Dec 2012 10:28, "Leigh Blackall" <leighblackall AT gmail.com> wrote:
Matt, I think you've unwittingly silenced an important thread, or the potential of it reaching important issues (if I do say so myself).
It could be the general tone has caused silence - I doubt it, it was always a quiet, gentle list.
Perhaps the CC NZ list is too politically sensitive for reckless debate, argument and controversy, I don't know who's enlisted, or if it's largely made up of people with unwillingness to raise a heart beat, or who feel they must be "careful" (silent or impotent).
I appreciate Danyl's consistent efforts to keep this list active and informed from his perspective. I don't mind a "rant", even if Paul would take the opportunity to belittle it, before sharing a valuable google search result.
I know my tone has heat at times, and may even be read as hostile, I admit to increasing frustration at the silence and non responsiveness of people with roles that should respond and engage more.
I do appreciate your willingness to moderate discussion though Matt, but without precision and recognition, you might simply serve to silence it.
On Nov 30, 2012 8:12 PM, "Matt McGregor" <Matt.McGregor AT royalsociety.org.nz> wrote:
Hi all,
There have been some productive and informative disagreements on the list--the NC and ND thread was a great example--where a range of people felt comfortable chipping in.
With that in mind, I think it's important that we keep the tone as cordial as possible. I'd hate for anyone to be discouraged from posting.
Enjoy your weekend,
Matt
________________________________________
From: cc-nz-bounces AT lists.ibiblio.org [cc-nz-bounces AT lists.ibiblio.org] on behalf of Sutherland, Paul [Paul.Sutherland AT ccc.govt.nz]
Sent: Thursday, November 29, 2012 9:17 PM
To: Creative Commons Aotearoa New Zealand discussion; Creative Commons Aotearoa New Zealand discussion
Subject: Re: [Cc-nz] Archiving the web for posterity
Hi
It is hard to know what is being asked here because the correspondent did not seem to do a basic google search to examine the landscape of the topic being addressed before the rant.
a search such as "web archive new zealand" http://goo.gl/Um4oD would bring up some key starting points
Wikipedia article List of Web archiving initiatives gives a rough global overview
http://en.wikipedia.org/wiki/List_of_Web_archiving_initiatives <http://en.wikipedia.org/wiki/List_of_Web_archiving_initiatives>
New Zealand web archive
http://natlib.govt.nz/collections/a-z/new-zealand-web-archive <http://natlib.govt.nz/collections/a-z/new-zealand-web-archive>
Web harvesting
http://natlib.govt.nz/publishers-and-authors/web-harvesting <http://natlib.govt.nz/publishers-and-authors/web-harvesting>
It is worth noting
The Internet Archive is a key player in the global internet archiving area and other tools have been developed for selective harvesting., The National Library of New Zealand is a global leader in this field, along with national libraries around the world
Regardless of what you think of any aspect of the Kim Dotcom saga you must remeber copyright is a legal instrument to protect the owners of intellectual property who have chosen to use this tool for whatever reason. MegaUpload facilitated the illegal duplication
and sharing of copyright material.
To enable the NLNZ to legally collect websites etc the New Zealand legislation needed to be changed to enable this.
This enabled a programme and the creation of the National Digital Heriatge Archive.
You tax dollars working for you
I hope that helps to answer some of your questions
/paul
________________________________
From: cc-nz-bounces AT lists.ibiblio.org on behalf of Danyl Strype
Sent: Thu 29/11/2012 8:14 p.m.
To: Creative Commons Aotearoa New Zealand discussion
Subject: [Cc-nz] Archiving the web for posterity
Kia ora koutou
Most of you will have heard of Archive.org (Internet Archive) and its
"Wayback Machine", which takes snapshots of websites and stores them,
like an archive storing issues of a newspaper on microfiche (I
remember learning to use a microfiche reader at school, and wondering
why anyone would store information in a way which was so ludicrously
difficult to access when it could be stored using computers - I didn't
know the half of it ;) This allows researchers, historians and other
interested people to look back for evidence in the published record,
without having to find a paper version in a physical archive, without
knowing in advance that they will need it (if you think you might want
a snapshot of a website for an academic citation, you can use
www.WebCitation.org).
The problem with the Wayback Machine though, is that it works pretty
much at random. Every now and then its 'robots' crawl your website,
making copies of the pages they can see.
Back in Web 1.0 prehistory, when websites consisted of an index of
flat HTML pages, which didn't change too often, this was fine. On a
web where some sites can change every second, and most sites are
actually complex server-side applications like G-Everything,
iEverything (patent pending), and MS Everything (look but don't
touch), using scripting languages like _javascript_, PHP, Python etc),
the Wayback Machine can only collect fragments and samples.
The Archive.org have attempted to solve this problem by offering a
site-specific archiving service for pages which are publicly
displayed. Archive-It allows webmasters to host their own archiving
robots, controlling what they can archive, and feeding the copies into
Archive.org:
http://www.archive-it.org/
Like Wikipedia, Archive.org is a USA-based, publicly-funded,
non-profit organisation. We can probably trust their motives, but we
certainly can't trust the neutrality of their government and its
enforcement machine, as the DotCom raids, and various other
copyright-based extradition attempts have shown (not to mention the
systematic attacks on WikiLeaks which US government "cyber-security"
agencies have either failed to investigate, or participated in). It
occurs to wonder if we have a publicly-funded digital archive doing
the same job for kiwi webpages that Archive.org does for those in the
US, using NZ-based servers which are "mirrored and striped" (across
multiple storage drives in multiple datacentres)? ArchivesNZ? National
Library? Digital NZ? Whose job is this, and is it getting done?
Hei kona
Strypey
--
Danyl Strype
Community Developer
Disintermedia.net.nz/strype
"Geeks are those who partake in our culture."
- .ISOcrates
"Uncomfortable alliances are not just necessary; they reflect and
speak to the tremendous possibility of our political moment."
- Harmony Goldberg and Joshua Kahn Russell
http://www.nationofchange.org/new-radical-alliances-new-era-1337004193
"Both Marxists and Chicago-school libertarian economists can agree
that free software is the best model."
- Keith C Curtis
http://keithcu.com/wordpress/?page_id=407
_______________________________________________
cc-nz mailing list
cc-nz AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/cc-nz
Creative Commons Aotearoa New Zealand
http://www.creativecommons.org.nz/
**********************************************************************
This electronic email and any files transmitted with it are intended
solely for the use of the individual or entity to whom they are addressed.
The views expressed in this message are those of the individual sender
and may not necessarily reflect the views of the Christchurch City Council.
If you are not the correct recipient of this email please advise the
sender and delete.
Christchurch City Council
http://www.ccc.govt.nz
**********************************************************************
_______________________________________________
cc-nz mailing list
cc-nz AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/cc-nz
Creative Commons Aotearoa New Zealand
http://www.creativecommons.org.nz/
_______________________________________________
cc-nz mailing list
cc-nz AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/cc-nz
Creative Commons Aotearoa New Zealand
http://www.creativecommons.org.nz/
_______________________________________________
cc-nz mailing list
cc-nz AT lists.ibiblio.org
http://lists.ibiblio.org/mailman/listinfo/cc-nz
Creative Commons Aotearoa New Zealand
http://www.creativecommons.org.nz/
