Skip to Content.
Sympa Menu

pcplantdb - Re: [pcplantdb] Talked with Jim this PM - update/info

pcplantdb@lists.ibiblio.org

Subject: pcplantdb

List archive

Chronological Thread  
  • From: Richard Morris <webmaster@pfaf.org>
  • To: pcplantdb@lists.ibiblio.org
  • Subject: Re: [pcplantdb] Talked with Jim this PM - update/info
  • Date: Thu, 13 May 2004 17:49:15 +0100

John Schinnerer wrote:
I told him you had put yourself forward as the one with the most
hardcore development chops and I don't recall anyone protesting that too
loudly.
I've a fair range of hardcore dev chops too!

You would be able to speak to some more nitty-gritty implementation details
that I can't - conceptually I'm feeling good about this project but to
actually code the concepts I and the rest of us are putting forth I'd have a
python learning curve about like El Capitan to climb.

Likewise, my python knowledge is non exsistant. Doesn't look too hard though.

> Not that I'm
adverse to
starting that climb, just more done for less time and/or money if you're
already summiting the climb...

If you really are not willing to talk with him in person at all, I want to
know why not, mainly because I want to know more about you if we are to be
working on this together.

I think the problem for me is that when I talked to Jim
it really brings up the questions which we have not thrashed
through ourselves yet. Do we share a tight enough vision of
what the project will be.

His main concern is how will we in practical terms dive in and proceed if he
gets us some money. The likely amount he mentioned coming through in July
was
in the $12k - $20k range, if it does. What could we do with that, he
wants to
know. He likes what we are about as he's heard it thus far.

This seems to be about the figure I was expecting,
much less than the original proposal, hence much
less salaried development time. I suspect this will
mean that we'll have to focus on the core elements
leaving the bells and whistles till later.
I guess this would encourage us towards a Keep it Simple
Stupid (KISS) development model.

I told him my take was that phase one would be a simple, practical,
PC-centric
plant database architected in such a way as to be able to grow from there.

Yep.

Focus would be on clean and robust back end, clean and open API and a simple
front end to start, with the clean and open API making all manner of front
ends possible eventually/as resources emerge.

Yep sounds good. The API does seem key. To some extent there
does seem to be seem to be a corespondence between API, XML-DTD
and database schema. I've read quite a bit on how there
can be fairly canonical mappings between the DTD and schema.
And the format of data returned by API calls (i.e. XML format)
is really part of the API.

It would serve initially as one answer to the inevitable PC-course
participants' question "...so where can I find permaculture-relevant
information on plant attributes and functions so I can use them appropriately
in designing??"

Yep, it would be good to know what our users want.

Yeah, I can see how an object DB would tie in neatly with an object
method of peer to peer data exchange.


I actually think it's a better model whether peer-to-peer or not.
Myself, I pitched this to Jim as strictly my take/my preference. He
seemed to
like the object based approach.

I've no real problems with an object based approach,
apart from never having played with it. I'm not
really sure on the advantages though; compare a
plant object
class Plant {
String LatinName;
String CommonName;
...
}

To a typical RDBS table

Table Plant
Field LatinName
Field CommonName
....

The one question I would have is if some binary format
object would be passed between front-end / backend.
To my mind text formats have a lot of advantages here:
1) Their intrinsically more open, you just
need to look at the data. Which format
would you perfere to work with HTML or
Word?
2) They might make things harder for clients
any client in any language can parse
XML or other text format. But will
MyFavoriteLanguage be able to interpret
objects created in YourFavoriteLanguage?

Again, if this is built carefully the guild aspect will be both implicit and
an emergent property. Some added logic to extract and/or extrapolate guild
information from plant objects and voila! Stephanie's dream realized! ;-)

Brain too small to understand this! Please expand.

I'm not opposed to the idea and
I'm totaly into the OO methodology, but I really don't know anything
about object DBs. Does zope have an object DB backend?

Yep, it's the ZODB (Z Object Database) and it's been stable and mature for
quite a while now. It has its own development track and releases and can be
(and has been) used directly (outside of Zope as a whole that is) to provide
persistence for python objects in all sorts of contexts.

Will have a look.

Available in conjunction with the ZODB is ZEO (Zope Enterprise Objects),
basically a scalable DB product that takes the default all-in-one ZODB and
makes it scalable across multiple servers. Pretty painlessly and
transparently too from what I understand. My sysadmin looked into it and was
impressed and he tends to be pretty skeptical and hard to impress.
We haven't implemented ZEO for our hosting yet because we simply don't need
it, but it's there when the need arises.

I doubt we'ed see a need for multiple servers in the short term.
(Different mirrors of same data maybe?)

The other nifty already-built piece available is ZCatalogs, which provide
cataloging of objects in a ZODB. You can define meta-data and indexes on
objects. Objects can be self-cataloging, so that when you instantiate
("add")
one it catalogs itself right then and there and will immediately appear in
any
relevant queries.

Could be very useful.

A ZODB can be indexed and queried using ZCatalogs and IME it is much clearer
and more obvious and less work than writing equivalently complex SQL queries
on an RDBMS.
From examples I have read it is "fast enough" for some pretty darn large
object DBs. Recent optimizations to the catalog code have cleaned up a
lot of
overhead and made it much faster than earlier versions.


What should I be reading/looking at? I also have some
concerns that may just be questions if I look into it more (like
performance).

I'm not sure if speeds/performance is going to be much of an
issue. The DB is small, 7000 entries, most things can
query that in less than a second.

Look up the ZODB, ZEO if you want for scalable DB options. These are rolled
into Zope as a whole, but if you google them or do a search on the zope.org
site you will find discrete info on them as separate entities. They are
afaik
in python with I think some C and/or C++ bits for performance reasons.

There are probably some other ODBs out there - this is just the one I use and
know a little something about.


I personally think XML is just the latest in a long list of useful but
tremendously over-hyped technologies.

Yah...

I don't think there is even that much to it as a technology.

For simple things its as good as any other text based format.
It can get really hairy when you start looking at
XPath, XLink, XML Schema, namespaces etc. etc. When you
get too involved in all the fancy bits it looses its
simplicity and the markup to data ratio goes sky high.

I thought there was way more to it than there is due to all the hype...I was
surprised to find out how relatively simple it is.
Which is good in the sense that it's more accessible to more people.
Anyhow, the hype generates active use of it and it's not IMO a 'bad'
technology, so more potential for growing the community of project
participants who might (for example) build a front end of some sort using the
XML in-between because it is familiar/comfortable for them.

I'd agree strongly here.

If someone is comfy with/into PHP and XML and we provide a clean and open API
from the back end then they will be more willing to do a front end to suit
them (or someone else).
I expect (but don't know) that it will also be readily usable by any sort of
graph-based front end.


I would be disappointed if we didn't get funded because he concluded
that we didn't have it together technically enough to make it happen.


His concerns are mostly (but not entirely) less technical and more
organizational was my impression. The technical is ultimately the easy part
in any project; it's the human relatings (cooperation, organization,
consensus
or not, etc.) that generate most of the difficulties. He knows that.
How will we organize ourselves, draw more project support (i.e. grow the
active contributor community), get started in a practical sense?

Yes these do seem to be the big issues.

Also, before I forget (we covered a LOT of ground on the phone), he is
curious
also about how we will manage input to the DB and had some good suggestions
based on his experience with social networks and so on. More than I can
cover
thoroughly now as I have to get to bed for a morning flight and will be
cyber-incommunicado until Tuesday next week.

In short we considered that different moderation approaches might be
appropriate for different content types, and Jim pointed to the value of
distributing/socializing the work of moderation.
We considered the difference between brief 'factual' data (climate zone,
shade
tolerance, soil preference, water needs, that sort of stuff) and less brief
'narrative' data (which is much richer and also harder to corroborate or
'verify' in a simple way).

What I'm doing for modoration of reader comments to PFAF DB
is that I get emailed a copy of every comment submitted.
It should be quite easy to do have a accept/reject email response.

To my mind such a system is all we'll really need for version
0.1. The rate of submissions will probably start small
(aprox 1 comment a day), so the workload will not be too hard.

The former could be for example vetted by 'qualified' community members in a
sort of distributed review process before being published publically. Who is
'qualified' is TBD and I won't go any deeper into that at the moment!

For the latter, we considered the possibility of a community-reputation
system
(like yes, sigh, eBay as a major example) where people can provide
feedback on
each other's narrative info. People who continually get called on submitting
questionable or bogus information will be visible as such by all users.
People who consistently submit useful and pertinent information will likewise
be visible as such by all users, and so on.
Jim pointed out also that this allows for the positive possibility of people
questioning and expanding the simple 'factual' data based on their own local
experiences.
Quick example - plant X is listed as not viable beyond zone 6 and this is
verified by multiple independent sources (other DBs, experienced
horticulturalists, USDA reference, etc.). Meanwhile someone somewhere has
been growing plant X successfully in zone 7 and can describe how they've done
so in a narrative submission - perhaps through the use of some clever design
that could be useful to others as well.

Sounds OK to me.

All for now - much food for thought - what next?

Finding out what the key questions we need to ask ourselves are?

Laters Rich
--
Plants for a Future: 7000 useful plants
Web: http://www.pfaf.org/ same as http://www.comp.leeds.ac.uk/pfaf/
Post: 1 Lerryn View, Lerryn, Lostwithiel, Cornwall, PL22 0QJ
Tel: 01208 872 963 / 0845 458 4719
Email: webmaster@pfaf.org
PFAF electronic mailing list http://groups.yahoo.com/group/pfaf







Archive powered by MHonArc 2.6.24.

Top of Page