Developer discussion for Creative Commons technology and tools

["Nathan R. Yergler" <nathan AT creativecommons.org>]

On 7/9/07, Jason Kivlighn <jkivlighn AT gmail.com> wrote:
> > Note that this script makes an incorrect assumption in determining if
> > a license has been superseded.  Namely it assumes that the versions
> > will be in ascending order in the XML file, which is not guaranteed.
> I'll fix that.
>
> I want to point out, though, that I don't picture this script as a
> long-term solution in generating licenses.  In my opinion, it hackishly
> pulls together files from all over, including translations from the po
> files in svn.

True enough, but for the time being the .xml file will remain the
canonical file.  Even when we move to using RDF it'll probably be
slightly different -- I'm unmotivated to  maintain tons of little
files when you can represent the entire mess with a single graph.


> I like the idea of migrating towards using these RDF license files in
> the other direction:  taking one source with all the license information
> and extracting the relevant information and using it to generate the
> translations, http://creativecommons.org/licenses/.../rdf, and so on.
>
> My 2 cents.
>
> Cheers,
> Jason
>
> >
> > NRY
> >
> >
> > On 7/5/07, jakin44 AT users.sourceforge.net
> > <jakin44 AT users.sourceforge.net> wrote:
> >> Revision: 6139
> >>           http://svn.sourceforge.net/cctools/?rev=6139&view=rev
> >> Author:   jakin44
> >> Date:     2007-07-05 22:34:16 -0700 (Thu, 05 Jul 2007)
> >>
> >> Log Message:
> >> -----------
> >> Script to generate RDF descriptions of all CC licenses.
> >>
> >> Added Paths:
> >> -----------
> >>     liblicense/gen_licenses.py
> >>
> >> Added: liblicense/gen_licenses.py
> >> ===================================================================
> >> --- liblicense/gen_licenses.py                          (rev 0)
> >> +++ liblicense/gen_licenses.py  2007-07-06 05:34:16 UTC (rev 6139)
> >> @@ -0,0 +1,252 @@
> >> +#!/usr/bin/env python
> >> +# Creative Commons has made the contents of this file
> >> +# available under a CC-GNU-GPL license:
> >> +#
> >> +# http://creativecommons.org/licenses/GPL/2.0/
> >> +#
> >> +# A copy of the full license can be found as part of this
> >> +# distribution in the file COPYING.
> >> +#
> >> +# You may use the liblicense software in accordance with the
> >> +# terms of that license. You agree that you are solely
> >> +# responsible for your use of the liblicense software and you
> >> +# represent and warrant to Creative Commons that your use
> >> +# of the liblicense software will comply with the CC-GNU-GPL.
> >> +#
> >> +# Copyright 2007, Creative Commons, www.creativecommons.org.
> >> +# Copyright 2007, Jason Kivlighn.
> >> +
> >> +from rdflib.Graph import Graph
> >> +from rdflib import Namespace, RDF
> >> +
> >> +from urllib2 import *
> >> +import xml.dom.minidom
> >> +import xml.dom.ext
> >> +import sys, os
> >> +
> >> +if not os.path.exists("i18n"):
> >> +       print "ERROR: Checkout
> >> https://svn.sourceforge.net/svnroot/cctools/i18n/trunk/i18n/ from svn
> >> into the current directory before running this script."
> >> +       sys.exit(1)
> >> +
> >> +try:
> >> +       os.mkdir("licenses")
> >> +except:
> >> +       pass
> >> +
> >> +header = """<!--
> >> +
> >> +Creative Commons has made the contents of this file
> >> +available under a CC-GNU-GPL license:
> >> +
> >> + http://creativecommons.org/licenses/GPL/2.0/
> >> +
> >> + A copy of the full license can be found as part of this
> >> + distribution in the file COPYING
> >> +
> >> +You may use the liblicense software in accordance with the
> >> +terms of that license. You agree that you are solely
> >> +responsible for your use of the liblicense software and you
> >> +represent and warrant to Creative Commons that your use
> >> +of the liblicense software will comply with the CC-GNU-GPL.
> >> +
> >> +Copyright 2007, Creative Commons, www.creativecommons.org.
> >> +
> >> +-->
> >> +"""
> >> +
> >> +PO_DIR="https://svn.sourceforge.net/svnroot/cctools/i18n/trunk/i18n/";
> >> 
> +LICENSE_FILE="https://svn.sourceforge.net/svnroot/cctools/license_xsl/trunk/licenses.xml";
> >>
> >> +
> >> +NS_DC = "http://purl.org/dc/elements/1.1/";
> >> +NS_DCQ = "http://purl.org/dc/terms/";
> >> +NS_RDF = "http://www.w3.org/1999/02/22-rdf-syntax-ns#";
> >> +
> >> +NS_CC = Namespace("http://creativecommons.org/ns#";)
> >> +
> >> +#x-default should be first in the list
> >> +xDefault = "en"
> >> +locales = [xDefault, "af","bg","ca","da","de","de_AT","de_CH","en_CA",
> >> +
> >> "en_GB","en_US","eo","es","es_AR","es_CL","es_CO","es_MX","es_PE",
> >> +
> >> "eu","fi","fr","fr_CA","fr_CH","fr_LU","gl","he","hr","hu","it","it_CH",
> >> +
> >> "ja","ko","kr","mk","ms","nl","nso","pl","pt","pt_PT","sl","st","sv",
> >> +       "zh","zh_TW","zu"]
> >> +
> >> +conn = urlopen(LICENSE_FILE)
> >> +license_xml = xml.dom.minidom.parse(conn)
> >> +
> >> +licenses = license_xml.getElementsByTagName('license')
> >> +print [license.getAttribute('id') for license in licenses]
> >> +for license in licenses:
> >> +       id = license.getAttribute('id')
> >> +       print id
> >> +       jurisdictions = license.getElementsByTagName('jurisdiction')
> >> +       for jurisdiction in jurisdictions:
> >> +               jurisdiction_id = jurisdiction.getAttribute('id')
> >> +               versions = jurisdiction.getElementsByTagName('version')
> >> +               replacedByURI =
> >> versions[len(versions)-1].getAttribute('uri')
> >> +               for version in versions:
> >> +                       version_id = version.getAttribute('id')
> >> +                       uri = version.getAttribute('uri')
> >> +
> >> +                       store = Graph()
> >> +
> >> +                       try:
> >> +                               store.load('%srdf' % uri)
> >> +                       except:
> >> +                               print uri, "failed"
> >> +                               pass
> >> +                       else:
> >> +                               doc = xml.dom.minidom.Document()
> >> +
> >> +                               rdf = doc.createElementNS(NS_RDF,
> >> "rdf:RDF")
> >> +                               doc.appendChild( rdf )
> >> +
> >> +                               description =
> >> doc.createElementNS(NS_RDF, "rdf:Description")
> >> +
> >> description.setAttributeNS(NS_RDF,"rdf:about",uri)
> >> +                               rdf.appendChild( description )
> >> +
> >> +                               if version_id != "-":
> >> +                                       hasVersion =
> >> doc.createElementNS(NS_DC, "dc:hasVersion")
> >> +                                       hasVersion.appendChild(
> >> doc.createTextNode(version_id) )
> >> +                                       description.appendChild(
> >> hasVersion )
> >> +
> >> +                               if len(versions) > 0 and
> >> replacedByURI != uri:
> >> +                                       isReplacedBy =
> >> doc.createElementNS(NS_DC, "dc:isReplacedBy")
> >> +                                       isReplacedByURI =
> >> doc.createElementNS(NS_DCQ, "dcq:URI")
> >> +                                       isReplacedByURIValue =
> >> doc.createElementNS(NS_RDF,"rdf:value")
> >> +
> >> isReplacedByURIValue.appendChild( doc.createTextNode(replacedByURI) )
> >> +                                       isReplacedByURI.appendChild(
> >> isReplacedByURIValue )
> >> +                                       isReplacedBy.appendChild(
> >> isReplacedByURI )
> >> +                                       description.appendChild(
> >> isReplacedBy )
> >> +
> >> +                               type = doc.createElementNS(NS_DC,
> >> "dc:type")
> >> +                               type.appendChild(
> >> doc.createTextNode("License") )
> >> +                               description.appendChild( type )
> >> +
> >> +                               identifier =
> >> doc.createElementNS(NS_DC, "dc:identifier")
> >> +                               identifier.appendChild(
> >> doc.createTextNode(uri) )
> >> +                               description.appendChild( identifier )
> >> +
> >> +                               if id in ["GPL","LGPL"]:
> >> +                                       creator_str = "Free Software
> >> Foundation"
> >> +                               elif id == "publicdomain":
> >> +                                       creator_str = None
> >> +                               else:
> >> +                                       creator_str = "Creative Commons"
> >> +
> >> +                               if creator_str:
> >> +                                       creator =
> >> doc.createElementNS(NS_DC, "dc:creator")
> >> +                                       creator.appendChild(
> >> doc.createTextNode(creator_str) )
> >> +                                       description.appendChild(
> >> creator )
> >> +
> >> +                               publisher =
> >> doc.createElementNS(NS_DC, "dc:publisher")
> >> +                               publisher.appendChild(
> >> doc.createTextNode("Creative Commons") )
> >> +                               description.appendChild( publisher )
> >> +
> >> +                               if jurisdiction_id != "-":
> >> +                                       coverage =
> >> doc.createElementNS(NS_DC, "dc:coverage")
> >> +                                       coverageCode =
> >> doc.createElementNS(NS_DCQ, "dcq:ISO3166")
> >> +                                       coverageCodeValue =
> >> doc.createElementNS(NS_RDF,"rdf:value")
> >> +
> >> coverageCodeValue.appendChild( doc.createTextNode(jurisdiction_id) )
> >> +                                       coverageCode.appendChild(
> >> coverageCodeValue )
> >> +                                       coverage.appendChild(
> >> coverageCode )
> >> +                                       description.appendChild(
> >> coverage )
> >> +
> >> +                                       basedOnURI =
> >> uri.rsplit("/",2)[0]+"/"
> >> +                                       isBasedOn =
> >> doc.createElementNS(NS_DC, "dc:isBasedOn")
> >> +                                       isBasedOnURI =
> >> doc.createElementNS(NS_DCQ, "dcq:URI")
> >> +                                       isBasedOnURIValue =
> >> doc.createElementNS(NS_RDF,"rdf:value")
> >> +
> >> isBasedOnURIValue.appendChild( doc.createTextNode(basedOnURI) )
> >> +                                       isBasedOnURI.appendChild(
> >> isBasedOnURIValue )
> >> +                                       isBasedOn.appendChild(
> >> isBasedOnURI )
> >> +                                       description.appendChild(
> >> isBasedOn )
> >> +
> >> +                               rdfType = doc.createElementNS(NS_RDF,
> >> "rdf:type")
> >> +
> >> 
> rdfType.setAttributeNS(NS_RDF,"rdf:resource","http://creativecommons.org/ns#License";)
> >>
> >> +                               description.appendChild( rdfType )
> >> +
> >> +                               for license in
> >> store.subjects(RDF.type, NS_CC["License"]):
> >> +                                       for requires in
> >> store.objects(license, NS_CC["requires"]):
> >> +                                               element =
> >> doc.createElementNS(NS_CC, "requires")
> >> +
> >> element.setAttributeNS(NS_RDF,"rdf:resource",str(requires))
> >> +
> >> description.appendChild( element )
> >> +                                       for permits in
> >> store.objects(license, NS_CC["permits"]):
> >> +                                               element =
> >> doc.createElementNS(NS_CC, "permits")
> >> +
> >> element.setAttributeNS(NS_RDF,"rdf:resource",str(permits))
> >> +
> >> description.appendChild( element )
> >> +                                       for prohibits in
> >> store.objects(license, NS_CC["prohibits"]):
> >> +                                               element =
> >> doc.createElementNS(NS_CC, "prohibits")
> >> +
> >> element.setAttributeNS(NS_RDF,"rdf:resource",str(prohibits))
> >> +
> >> description.appendChild( element )
> >> +
> >> +                               #try:
> >> +
> >> +                               translation_map = {}
> >> +
> >> +                               title = doc.createElementNS(NS_DC,
> >> "dc:title")
> >> +                               title_alt =
> >> doc.createElementNS(NS_RDF, "rdf:Alt")
> >> +                               title.appendChild( title_alt )
> >> +                               description.appendChild(title)
> >> +                               if id == "devnations":
> >> +                                       translation_map['msgid
> >> "util.Developing_Nations"'] = title_alt
> >> +                               else:
> >> +                                       translation_map['msgid
> >> "licenses.pretty_%s"' % id] = title_alt
> >> +
> >> +                               """
> >> +                               dcDescription =
> >> doc.createElementNS(NS_DC, "dc:description")
> >> +                               dcDescription_alt =
> >> doc.createElementNS(NS_RDF, "rdf:Alt")
> >> +                               dcDescription.appendChild(
> >> dcDescription_alt )
> >> +                               description.appendChild(dcDescription)
> >> +                               translation_map['msgid
> >> "char.%s_description"' % id] = dcDescription_alt
> >> +                               """
> >> +
> >> +                               for locale in locales:
> >> +                                       try:
> >> +                                               #conn =
> >> urlopen(PO_DIR+"icommons-%s.po" % locale)
> >> +                                               conn =
> >> open("i18n/icommons-%s.po" % locale,"r")
> >> +                                               lines = conn.readlines()
> >> +                                               i = 0
> >> +                                               while i < len(lines):
> >> +                                                       line =
> >> lines[i].decode("utf8").strip()
> >> +                                                       element =
> >> translation_map.get(line)
> >> +                                                       if element:
> >> +                                                               i += 1
> >> +
> >> msgstr = lines[i].strip().lstrip('msgstr "').rstrip('"')
> >> +                                                               li =
> >> doc.createElementNS(NS_RDF,"rdf:li")
> >> +                                                               if
> >> locale == xDefault:
> >> +
> >> li.setAttributeNS(xml.dom.XML_NAMESPACE,"xml:lang","x-default")
> >> +                                                               else:
> >> +
> >> if locale.find("_") == -1:
> >> +
> >> locale = locale + "_" + locale.upper()
> >> +
> >> li.setAttributeNS(xml.dom.XML_NAMESPACE,"xml:lang",locale)
> >> +
> >> +                                                               if
> >> creator_str == "Creative Commons":
> >> +
> >> msgstr = "Creative Commons - "+msgstr
> >> +
> >> li.appendChild( doc.createTextNode(msgstr) )
> >> +
> >> +
> >> element.appendChild( li )
> >> +                                                       i += 1
> >> +                                       except:
> >> +                                               print "getting po
> >> file, %s, failed" % locale
> >> +                                               continue
> >> +
> >> +                               output = "licenses/%s.rdf" %
> >> uri.lstrip("http://";).replace("/","_")
> >> +                               output_file = open(output,"w")
> >> +                               #doc.writexml(output_file,'','\t','\n')
> >> +                               xml.dom.ext.PrettyPrint(doc,output_file)
> >> +                               output_file.close()
> >> +
> >> +                               # Now tack on the header comment
> >> +                               f_out = open(output+".new","w")
> >> +                               f_in = open(output,"r")
> >> +
> >> +                               f_out.write(f_in.readline())
> >> +                               f_out.write(header)
> >> +                               f_out.write(f_in.read())
> >> +
> >> +                               f_out.close()
> >> +                               f_in.close()
> >> +
> >> +                               os.rename(output+".new",output)
> >> +
> >> +print "Licenses output to licenses/"
> >>
> >>
> >> Property changes on: liblicense/gen_licenses.py
> >> ___________________________________________________________________
> >> Name: svn:executable
> >>    + *
> >>
> >>
> >> This was sent by the SourceForge.net collaborative development
> >> platform, the world's largest Open Source development site.
> >>
> >> -------------------------------------------------------------------------
> >>
> >> This SF.net email is sponsored by DB2 Express
> >> Download DB2 Express C - the FREE version of DB2 express and take
> >> control of your XML. No limits. Just data. Click to get it now.
> >> http://sourceforge.net/powerbar/db2/
> >> _______________________________________________
> >> Cctools-cvs mailing list
> >> Cctools-cvs AT lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/cctools-cvs
> >>
> >
>
> _______________________________________________
> cc-devel mailing list
> cc-devel AT lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/cc-devel