Developer discussion for Creative Commons technology and tools

["Hugo Dworak" <uninhabitability AT gmail.com>]

Dear cc-devel Subscribers,

Google Summer of Code 2008 approaches its end, as less than
forty-eight hours are left to submit the code that will then be
evaluated by mentors, therefore it is fitting to pause for a moment
and sum up the work that has been done with regard to the
license-oriented metadata validator and viewer and to confront it with
the original proposal for the project.

A Web application capable of parsing and displaying license
information embedded in both well-formed and ill-formed Web pages has
been developed. It supports the following means of embedding license
information: Dublin Core metadata, RDFa, RDF/XML linked externally or
embedded (utilising the ``data'' URL scheme) using the "link" and "a"
elements, and RDF/XML embedded in a comment or as an element (the last
two being deprecated). This functionality has been proven by unit
testing. The source code of a Web page can be uploaded or pasted by a
user, there is also a possibility to provide a URI for the Web
application to analyse it. The software has been written in Python and
uses the Pylons Web Framework and the Genshi toolkit. Should you be
willing to test this Lynx-friendly application, please visit its Web
site <http://validator-beta.creativecommons.org/>.

The Web application itself uses a library called ``libvalidator'',
which in turn is powered by cc.license (a library developed by
Creative Commons that returns information about a given license),
pyRdfa (a distiller that generates the RDF triples from an
(X)HTML+RDFa file), html5lib (an HTML parser/tokenizer), and RDFLib (a
library for working with RDF). The choice of this set of tools has not
been obvious and the library had undergone several redesigns, which
included removing the code that employed encutils, XML
canonicalization, uTidylib, and the BeautifulSoup. The idea of using
librdf, librdfa, rdfadict has been abandoned. The source code of both
the Web application (licensed under the GNU Affero General Public
License version 3 or newer) and its core library (licensed under the
GNU Lesser General Public License version 3 or newer) is available
through the Git repositories of Creative Commons.

In contrast to the contents of the original proposal, the following
goals have not been met: traversal of special links, syndication feeds
parsing, statistics, and cloning the layout of the Creative Commons
Web site. However, these were never mandatory requirements for the Web
application. It is also worth noting that the software has been
written from scratch, although a now-defunct metadata validator
existed. Nevertheless, the development does not end with Google Summer
of Code --- these and several new features (such as validation of
multimedia files via liblicense and support for different language
versions) are planned to be added, albeit at a slower pace.

After the test period, the validator will be available under
<http://validator.creativecommons.org/>.

Sincerely,

--
Hugo Dworak

http://hugo.dworak.info/