Permaculture Database

[Matthew Shultz <deepwinter@winterroot.net>]

Hey Jedd,
It might be interesting to get on the phone sometime.  What time zone are you 
in?

Key-Value store is bad in a lot of ways for querying, I agree.  But what i'm 
suggesting in some ways is a 2 phase approach.  A key-value store is great 
for letting people log a ton of data, all kinds of data that we might not 
even have to know they would have beforehand (and it's very hard to think out 
a normalized schema when we're talking literally hundreds of possible 
parameters).   However later on, when there is data available, it's 
straightforward to transform elements of a key-value store into properly 
normalized tables and load them into a RDBMS for querying.  My thought is 
that a key value store is perfect for sort of a shared log book, and then the 
keys that get used be organized down the road for pattern search.  Get my 
gist?  I can send along some sketches of how I'm thinking a simple interface 
would work for this.

m.

On Feb 19, 2011, at 3:33 PM, jedd wrote:

> Hi Matthew,
> 
> I haven't been slack for some self-fulfilling prophecy here.  ;)
> 
> I've been working - in my day job - with Cassandra, a massively
> scalable key-value store.  In the recently released v0.7.x
> branch, it can be a bit smarter with indexes, but it's still a long
> way from being anywhere near as useful (for what I want to do,
> or think I want to do) as a SQL-able RDBMS.
> 
> I'm starting to look at SciDB - it has a pretty impressive set of
> goals, and an even more impressive progenitor driving it - but
> is some way off being ready for production I think.
> 
> OTOH, the amount of data that I'm thinking about will, or at
> least the key set will, easily sit inside a reasonable amount of
> memory - so I'm in no hurry to find esoteric solutions for that
> aspect of my problems.  OTOOH I may well be grossly under-
> estimating the size of the problems that I may one day have.
> 
> What is it they say?  Algorithms are for people who can't afford RAM.
> 
> I'd love to know a bit more about this VSAS thing - can't find any
> obvious candidates on the intergoogles - but obdisclaimer here is
> that I'm hip with the basics, and only the basics, of OO.
> 
> On the upside, I'm a big fan of CI, so the Kohana layout should be
> easy to accustom to.  What's the licence on your work?
> 
> Sorry to hear about the iPhone app.  ;)   (Nokia N900 user here,
> though sadly my street cred has fallen through the floor since
> Nokia sidled up to Microsoft.)
> 
> My gut feel, as ever, with statistics for the average permaculturist,
> the big problem will be getting enough raw data into the system in
> a useful format.  The second will be having a system free and
> designed and documented sufficiently to let (other) people do
> the really interesting statistical work.
> 
> The permy list in oceania has, of late, become pretty depressing
> in terms of in-fighting and the generation of closed or exclusive
> systems.  I hope it's a blip.
> 
> j.
> _______________________________________________
> pcdb mailing list
> pcdb@lists.ibiblio.org
> http://lists.ibiblio.org/mailman/listinfo/pcdb