[cc-devel] Metadata scraper roadmap

Maarten Zeinstra mz at kl.nl
Thu Feb 28 08:23:47 EST 2013

Hi all,

My rant was not directed at Greg personally, I've mailed him off list and he understands that. My frustration and fear is that Creative Commons is nearing a state of technological ruin, only having a great groundskeeper at the moment. (That is a compliment Nathan ;) )

All affiliates and CC-activitsts base their reputation and success partly on that infrastructure. I would be very much in favour for a more decentralised system of development and support. But to be able to enable that we need to have a strong core that is maintained and developed regardless.

For example, there is no public overview of how different application, services are run and how they are run. How many servers (virtual and real) are there to be maintained? How do all these products interconnect and depend on each other?  

Like Sylvain I can also contribute to the code base and other projects like I tried in the past. What we really need is someone who coordinates, writes roadmaps, and integrates contributions. It is not that we don't care we simply can't get any support from a product owner.



On Feb 28, 2013, at 13:40 , Sylvain Zimmer <sylvain at sylvainzimmer.com> wrote:

> As a crawler-enthusiast, I'd be willing to give a hand if given clear pointers on the current infrastructure and future needs!
> Cheers,
> --
> Sylvain Zimmer
> Founder, Jamendo
> blog: sylvinus.org
> gsm: +33 6 64 67 61 71
> On Thu, Feb 28, 2013 at 1:33 PM, Jonas Öberg <jonas at coyote.org> wrote:
> I too think it's important that work continues on metadata standards
> and other infrastructure work, but also that this can not be the work
> of Creative Commons alone, as an organisation. As you say, it would be
> very useful to see the ecosystem around this include other
> organisations and individuals. It would create for a stronger alliance
> around the important infrastructure we need to support the commons,
> technically, in the future.
> What would be a suitable way forward? I know that CC in the past
> organised technology summits that I felt was useful. Still, it would
> be good to make progress without needing to physically get together
> for something to happen :)
> Best
> Jonas
> On Thu, Feb 28, 2013 at 12:14 AM, Mike Linksvayer <ml at gondwanaland.com> wrote:
> > Maarten,
> >
> > You're right about everything below.
> >
> > But Greg is not a person to beat up on. He doesn't work for CC now,
> > and his responsibility there was (as mandated by a grant) LRMI. I'm
> > certain he's sincere in that he wishes he had better answers. CC
> > hasn't had any tech leadership since Nathan Yergler left nearly 2
> > years ago, and the months before him leaving were transitional. Now it
> > has no software engineers either. PDM (launched close to 2.5 years
> > ago) is the last big thing done (but kudos to those who pushed the
> > updated chooser out before exiting in the past year).
> >
> > I've been advising CC most of this period, and take some
> > responsibility for failing to press for hiring a CTO and more
> > engineers. Others, including Greg, tried harder.
> >
> > I'm rooting for contemporary CC to do better, and maybe your emails
> > will help. But I also hope that some entities outside of CC the
> > organization build and improve tools.That would be a healthier
> > ecosystem, even if the ultimate aim is just for the centralized
> > metadata scraper to work better (which is an absurdly low aim, and yes
> > absurd that it is not reached) -- expensive things relied on by few
> > generally don't get improved -- that's metadata in a nutshell. It's
> > really amazing over the years how few people have actively complained
> > about CC "tech tools" and documentation (not counting "search is a
> > problem!!!") -- it's largely because almost nothing is relying on them
> > end to end, the deed scraper being maybe the only active exception,
> > initially built in part to have something that actually consumed
> > recommended metadata.
> >
> > Mike
> >
> > On Wed, Feb 27, 2013 at 2:40 PM, Maarten Zeinstra <mz at kl.nl> wrote:
> >> Oh come on!
> >>
> >> That simply is not good enough.
> >>
> >> We CC-affiliates / open activists need to be able to do our work. I raised this question because I do work for Europeana, The European version of the DPLA, but 5 years ahead. they were instrumental and the first adopter of the Public Domain Mark. Europeana works with 2600+ cultural institutions and has about 5 million works PD/CC0 marked. Now when one of those institutions mails me and asked me why there is no extra metadata on PDM pages coming of Europeana, I need to be able to say that there is possibly a script blocker on the client side and that Europeana and CreativeCommons.org are functioning properly, see this page for more information (no page or documentation to be found...). When I cannot do that I cannot do my job. When I cannot do my job I cannot convince these institutions of the merit of CC licensing or PD marking. When I cannot do that they will be less likely to use the tools. That is bad for our global access to culture. (I use myself as an example, this could happen to anyone who wants to promote CC).
> >>
> >> We all want more open content and correct rights labelling right? I need to be able to rely on that infrastructure of CC.org to be able to do my job in this.
> >>
> >> It is completely absurd that you cannot get your act together and provide us with a decent technological infrastructure and support to help convince the world that a) open content is a good way to go and b) rights labelling is important and c) (most importantly) we have their backs in that.
> >>
> >> Also, where is Dan Mills in all of this? He's been on the job for like two months now right? Why is it that he hasn't even introduced himself on this list yet? I hope he consider the licenses and their backing technology as part of the product of CC..
> >>
> >> I think it is unbelievable that you can laconically state that you wish you had better answers…
> >>
> >>
> >> On Feb 27, 2013, at 18:15 , Greg Grossmeier <greg at grossmeier.net> wrote:
> >>
> >>> <quote name="Maarten Zeinstra" date="2013-02-27" time="12:18:05 +0100">
> >>>> I was wondering if there is a roadmap for the development of the
> >>>> metadata scraper of the deed pages.
> >>>
> >>> This is on a roadmap of some sorts; I can't remember the specific
> >>> timeline right now.
> >>>
> >>>> As of now it only support RDFa, maybe we want to add other formats as
> >>>> well?
> >>>
> >>> That is the idea, indeed.
> >>>
> >>>> Also I could find very little documentation about the workings of the
> >>>> scraper directed at non-developers. Where can I point to when no
> >>>> scraped information shows up because of a script blocker?
> >>>
> >>> Nothing at this time :/
> >>>
> >>>
> >>> Wish I had better answers,
> >>>
> >>> Greg
> >>>
> >>>
> >>> PS: As of Feb 19th, I now work for the Wikimedia Foundation, so my time
> >>> on these type of issues will be radically lower now.
> >>> http://lists.wikimedia.org/pipermail/wikitech-l/2013-February/066672.html
> >>>
> >>> --
> >>> | Greg Grossmeier            GPG: B2FA 27B1 F7EB D327 6B8E |
> >>> | http://grossmeier.net           A18D 1138 8E47 FAC8 1C7D |
> >>> _______________________________________________
> >>> cc-devel mailing list
> >>> cc-devel at lists.ibiblio.org
> >>> http://lists.ibiblio.org/mailman/listinfo/cc-devel
> >>
> >> _______________________________________________
> >> cc-devel mailing list
> >> cc-devel at lists.ibiblio.org
> >> http://lists.ibiblio.org/mailman/listinfo/cc-devel
> > _______________________________________________
> > cc-devel mailing list
> > cc-devel at lists.ibiblio.org
> > http://lists.ibiblio.org/mailman/listinfo/cc-devel
> _______________________________________________
> cc-devel mailing list
> cc-devel at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/cc-devel
> _______________________________________________
> cc-devel mailing list
> cc-devel at lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/cc-devel

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.ibiblio.org/pipermail/cc-devel/attachments/20130228/7aec77e6/attachment.html 

More information about the cc-devel mailing list