pcplantdb

[Richard Morris <webmaster@pfaf.org>]

John Schinnerer wrote:
> I told him you had put yourself forward as the one with the most
> hardcore development chops and I don't recall anyone protesting that too
> loudly.
I've a fair range of hardcore dev chops too!

> You would be able to speak to some more nitty-gritty implementation details
> that I can't - conceptually I'm feeling good about this project but to
> actually code the concepts I and the rest of us are putting forth I'd have a
> python learning curve about like El Capitan to climb.  

Likewise, my python knowledge is non exsistant. Doesn't look too hard 
though.

> Not that I'm
> adverse to
> starting that climb, just more done for less time and/or money if you're
> already summiting the climb...
>
> If you really are not willing to talk with him in person at all, I want to
> know why not, mainly because I want to know more about you if we are to be
> working on this together.
I think the problem for me is that when I talked to Jim
it really brings up the questions which we have not thrashed
through ourselves yet. Do we share a tight enough vision of
what the project will be.

> His main concern is how will we in practical terms dive in and proceed if he
> gets us some money.  The likely amount he mentioned coming through in July
> was
> in the $12k - $20k range, if it does.  What could we do with that, he
> wants to
> know.  He likes what we are about as he's heard it thus far.

This seems to be about the figure I was expecting,
much less than the original proposal, hence much
less salaried development time. I suspect this will
mean that we'll have to focus on the core elements
leaving the bells and whistles till later.
I guess this would encourage us towards a Keep it Simple
Stupid (KISS) development model.

> I told him my take was that phase one would be a simple, practical,
> PC-centric
> plant database architected in such a way as to be able to grow from there.

Yep.

> Focus would be on clean and robust back end, clean and open API and a simple
> front end to start, with the clean and open API making all manner of front
> ends possible eventually/as resources emerge.

Yep sounds good. The API does seem key. To some extent there
does seem to be seem to be a corespondence between API, XML-DTD
and database schema. I've read quite a bit on how there
can be fairly canonical mappings between the DTD and schema.
And the format of data returned by API calls (i.e. XML format)
is really part of the API.

> It would serve initially as one answer to the inevitable PC-course
> participants' question "...so where can I find permaculture-relevant
> information on plant attributes and functions so I can use them appropriately
> in designing??"
Yep, it would be good to know what our users want.
> > Yeah, I can see how an object DB would tie in neatly with an object
> > method of peer to peer data exchange.
>
>
> I actually think it's a better model whether peer-to-peer or not.
> Myself, I pitched this to Jim as strictly my take/my preference.  He
> seemed to
> like the object based approach.

I've no real problems with an object based approach,
apart from never having played with it. I'm not
really sure on the advantages though; compare a
plant object
        class Plant {
                String LatinName;
                String CommonName;
                ...
        }

To a typical RDBS table

Table Plant
        Field LatinName
        Field CommonName
        ....

The one question I would have is if some binary format
object would be passed between front-end / backend.
To my mind text formats have a lot of advantages here:
        1) Their intrinsically more open, you just
                need to look at the data. Which format
                would you perfere to work with HTML or
                Word?
        2) They might make things harder for clients
                any client in any language can parse
                XML or other text format. But will
                MyFavoriteLanguage be able to interpret
                objects created in YourFavoriteLanguage?

> Again, if this is built carefully the guild aspect will be both implicit and
> an emergent property.  Some added logic to extract and/or extrapolate guild
> information from plant objects and voila!  Stephanie's dream realized!  ;-)
Brain too small to understand this! Please expand.
> > I'm not opposed to the idea and
> > I'm totaly into the OO methodology, but I really don't know anything
> > about object DBs.  Does zope have an object DB backend?
>
> Yep, it's the ZODB (Z Object Database) and it's been stable and mature for
> quite a while now.  It has its own development track and releases and can be
> (and has been) used directly (outside of Zope as a whole that is) to provide
> persistence for python objects in all sorts of contexts.

Will have a look.

> Available in conjunction with the ZODB is ZEO (Zope Enterprise Objects),
> basically a scalable DB product that takes the default all-in-one ZODB and
> makes it scalable across multiple servers.  Pretty painlessly and
> transparently too from what I understand.  My sysadmin looked into it and was
> impressed and he tends to be pretty skeptical and hard to impress.
> We haven't implemented ZEO for our hosting yet because we simply don't need
> it, but it's there when the need arises.

I doubt we'ed see a need for multiple servers in the short term.
(Different mirrors of same data maybe?)

> The other nifty already-built piece available is ZCatalogs, which provide
> cataloging of objects in a ZODB.  You can define meta-data and indexes on
> objects.  Objects can be self-cataloging, so that when you instantiate
> ("add")
> one it catalogs itself right then and there and will immediately appear in
> any
> relevant queries.

Could be very useful.

> A ZODB can be indexed and queried using ZCatalogs and IME it is much clearer
> and more obvious and less work than writing equivalently complex SQL queries
> on an RDBMS.
> > From examples I have read it is "fast enough" for some pretty darn large
> object DBs.  Recent optimizations to the catalog code have cleaned up a
> lot of
> overhead and made it much faster than earlier versions.
>
>
> > What should I be reading/looking at?  I also have some
> > concerns that may just be questions if I look into it more (like
> > performance).
I'm not sure if speeds/performance is going to be much of an
issue. The DB is small, 7000 entries, most things can
query that in less than a second.
> Look up the ZODB, ZEO if you want for scalable DB options.  These are rolled
> into Zope as a whole, but if you google them or do a search on the zope.org
> site you will find discrete info on them as separate entities.  They are
> afaik
> in python with I think some C and/or C++ bits for performance reasons.
>
> There are probably some other ODBs out there - this is just the one I use and
> know a little something about.
>
>
> > I personally think XML is just the latest in a long list of useful but
> > tremendously over-hyped technologies.
>
> Yah...
>
> > I don't think there is even that much to it as a technology.
For simple things its as good as any other text based format.
It can get really hairy when you start looking at
XPath, XLink, XML Schema, namespaces etc. etc. When you
get too involved in all the fancy bits it looses its
simplicity and the markup to data ratio goes sky high.

> I thought there was way more to it than there is due to all the hype...I was
> surprised to find out how relatively simple it is.
> Which is good in the sense that it's more accessible to more people.
> Anyhow, the hype generates active use of it and it's not IMO a 'bad'
> technology, so more potential for growing the community of project
> participants who might (for example) build a front end of some sort using the
> XML in-between because it is familiar/comfortable for them.

I'd agree strongly here.

> If someone is comfy with/into PHP and XML and we provide a clean and open API
> from the back end then they will be more willing to do a front end to suit
> them (or someone else).
> I expect (but don't know) that it will also be readily usable by any sort of
> graph-based front end.
>
>
> > I would be disappointed if we didn't get funded because he concluded
> > that we didn't have it together technically enough to make it happen.
>
>
> His concerns are mostly (but not entirely) less technical and more
> organizational was my impression.  The technical is ultimately the easy part
> in any project; it's the human relatings (cooperation, organization,
> consensus
> or not, etc.) that generate most of the difficulties.  He knows that.
> How will we organize ourselves, draw more project support (i.e. grow the
> active contributor community), get started in a practical sense?
Yes these do seem to be the big issues.

> Also, before I forget (we covered a LOT of ground on the phone), he is
> curious
> also about how we will manage input to the DB and had some good suggestions
> based on his experience with social networks and so on.  More than I can
> cover
> thoroughly now as I have to get to bed for a morning flight and will be
> cyber-incommunicado until Tuesday next week.
>
> In short we considered that different moderation approaches might be
> appropriate for different content types, and Jim pointed to the value of
> distributing/socializing the work of moderation.
> We considered the difference between brief 'factual' data (climate zone,
> shade
> tolerance, soil preference, water needs, that sort of stuff) and less brief
> 'narrative' data (which is much richer and also harder to corroborate or
> 'verify' in a simple way).

What I'm doing for modoration of reader comments to PFAF DB
is that I get emailed a copy of every comment submitted.
It should be quite easy to do have a accept/reject email response.

To my mind such a system is all we'll really need for version
0.1. The rate of submissions will probably start small
(aprox 1 comment a day), so the workload will not be too hard.

> The former could be for example vetted by 'qualified' community members in a
> sort of distributed review process before being published publically.  Who is
> 'qualified' is TBD and I won't go any deeper into that at the moment!
>
> For the latter, we considered the possibility of a community-reputation
> system
> (like yes, sigh, eBay as a major example) where people can provide
> feedback on
> each other's narrative info.  People who continually get called on submitting
> questionable or bogus information will be visible as such by all users.
> People who consistently submit useful and pertinent information will likewise
> be visible as such by all users, and so on.
> Jim pointed out also that this allows for the positive possibility of people
> questioning and expanding the simple 'factual' data based on their own local
> experiences.
> Quick example - plant X is listed as not viable beyond zone 6 and this is
> verified by multiple independent sources (other DBs, experienced
> horticulturalists, USDA reference, etc.).  Meanwhile someone somewhere has
> been growing plant X successfully in zone 7 and can describe how they've done
> so in a narrative submission - perhaps through the use of some clever design
> that could be useful to others as well.

Sounds OK to me.

> All for now - much food for thought - what next?

Finding out what the key questions we need to ask ourselves are?

        Laters Rich
--
Plants for a Future: 7000 useful plants
Web:   http://www.pfaf.org/ same as http://www.comp.leeds.ac.uk/pfaf/
Post:  1 Lerryn View, Lerryn, Lostwithiel, Cornwall, PL22 0QJ
Tel:     01208 872 963 / 0845 458 4719  
Email: webmaster@pfaf.org
PFAF electronic mailing list http://groups.yahoo.com/group/pfaf