pcplantdb

[Chad Knepp <pyg@galatea.org>]

Hey there everyone,

I'm including a draft of the report.  Please edit and repost if you
have changes/additions/corrections.  I also want to weigh in a little
on some stuff Bear and Stephanie have said recently.

As an update our current plans, IHMO, are to clean up the version on
dev (0.3.1) and put it onto live.  Most of this work is cosmetic UI
stuff... I'm actually beginning to believe that there aren't really
very many actual bugs in 0.3.1... at least stable enough to run live.
This isn't that much work.  I would say less than 20 hours or so.
This is when I would like to annouce it to the ibiblio list and let a
larger user base bang on it a bit (release early and release often)
with appropriate caveats for developmental software.  After that I
would like to implement some features off the clock and then resume
finishing our commitment to Threshold to be DONE by end of 2006 at
least.


Bear Kaufmann writes:
 > Hi all,
 > 
 > I've been working on a mashup recently. I'll jump on the Web 2.0 jargon. 
 > Though I'll have to say Web 2.0 is a little over hyped now.
 > Anyway, it pulls from Google Images, 'Duke's Phytochemical and 
 > Ethnobotanical Database', International Plant Names (Inst?) IPNI, GRIN, 
 > links to Wikispecies (family, genus, species), and will pull from USDA 
 > plants on the fly.
 > It's still not perfect, so I'm not going to up load it yet. I'd also 
 > need to figure out what's fair use and/or proper citation for each 
 > source. I was inspired by the mashup on ispecies.org.
 > 
 > I've also been working Actionscript. There's an Actionscript compiler 
 > (Unix bin) which works real well with XCode called MTASC, which 
 > generates SWF (Flash) files. Honestly, I tried to get TouchGraph up and 
 > compilable in Eclipse and also XCode, and Java was not treating me well. 
 > It's not my native tongue, but I'm picking up Actionscript quickly. I'm 
 > working on adapting an open source physics engine to implement something 
 > akin to TouchGraph for the RichClient. I'll show you something when it's 
 > presentable. Plus the current compiled file is 8K, not that it'll remain 
 > 8K, but Flash is optimized for bandwidth and graphics.

Sounds cool, got links to [unix] Actionscript stuff?

 > Lastly, I wouldn't mind seeing some of the grant money to pay for my 
 > time. Putting it towards a laptop would give me a bit more time to code 
 > while I'm commuting this semester.

I've been telling you for sometime to bill for your design work.  Your
work is certainly worth it.  I think it's great that you (and Rich)
are staying in touch with all the hot meta-data technologies and I
certainly expect to add OWL, RDF, RSS, whatever... but not right now.
I really want to get done with what we said we would do before moving
on to other things, or do new things on a purely voluntary basis (such
as me implementing the features images, moderation, and articles off
the clock just because I want them).  I'm guessing that making PIW
compatible with any of the technologies is also going to be pretty
easy to do esp. on the server level (for example xml-rpc).

I'm a little hesitant to say that we should use the funding money to
get hardware for ourselves.  If we did really want to do this, I would
want to be in line as well.  My hardware consists of free
hand-me-downs from relatives with nothing less than six years old.

I would really really like it if you would do more design work on
dev.  I would feel comfortable with budgeting 100 hours or so to get
you to work more magic on things like making the forms more
friendly/consistent, better UI stuff (it's all mostly my mess right
now), more consistent layout/css between static pages, search results,
and plant reports.  Interested?  Working on GBI stuff is still part of
our commitment as well and I think we should start paying for time
developing it so we can get it down.  I know that Rich was also
interested in this aspect.  You or Rich (or both) interested?

Cheers,
Chad

-- 
Chad Knepp
python -c 'import base64;print base64.decodestring("cHlnQGdhbGF0ZWEub3Jn")'



The Permaculture Information Web 2005 Report

Results

History:

Beginning in August of 2004 we explored creating an object database
based Zope/ZODB, but after some simple prototyping abandoned it due to
unneeded complexity, overhead, and bugs.  In October of 2004 we began
development based on the Plants For A Future dataset and a simple
interface to it called Eden.  Development and discussion has contined
since then.  In August of 2005 we released a fully editable version
(0.3.1).

Features of version 0.3.1:

Although modified from the original PFAF database, one of the most
significant feature additions so far is that almost all elements of
the database are editable. It is now possible to add new plants and
add/edit descriptions, uses, optimal locations, comments/notes of
additional details, as well as plant cultivars and comments/notes
about cultivars.

PIW also has a well developed search engine that allows both a simple
text string search of most of the text based data as well as an
advanced search with 100+ options that allow a focused search over
very specific criteria.  Both searches return results with bar graphs
indicating search relevance.  Plant reports are customizable within
the client, although capability to adjust preferences is currently
unimplemented.

The software architecture of PIW is a classic server/client.  The
server is comparably quite small and is reliability/security oriented.
It exposes its methods to clients via the XML-RPC protocol and its API
is documented on <http://dev.permaculture.info/api/>.  The primary
client is a cgi script that, for performance reasons, does not use
xml-rpc but directly imports the server packages.  This web based HTML
generating client comprises the bulk of the PIW code.  The advantage
of xml-rpc is that it allows clients to be written in any
language/platform that supports xml-rpc.  In addition to custom
clients, this will allow assimilation of other datasets via custom
scripts that interface directly with the server.  One of our
developers has written some custom search/report scripts in PHP,
although the server is written in python.  Unfortunately, limitations
imposed by our ISP do not allow us to run a direct server, so we are
using a cgi to xml-rpc gateway.  You can demonstrate this with the
following python code which arbitrarily retrieves comment #8000:

#!/usr/bin/env python
import xmlrpclib
server = 
xmlrpclib.ServerProxy("http://dev.permaculture.info/cgi-bin/eden_xml-rpc.py";)
print server.GetComment(8000)

You should get the following results:

{'CommentText': "Fruit - raw or cooked. It contains a number of small
seeds, but these are easily eaten with the fruit[<link user='Ken
Fern'>K</link>].", 'Author': 'Ken Fern', 'Title': 'Edible uses',
'Modified': '2005-07-04 08:41:44', 'ParentType': 'plant', 'ParentId':
175, 'Id': 8000}

A careful perusal of the above results will show some of our internal
format xml (kfml).  Although it currently only contains a few tags,
they are a very portable way of interlinking the dataset within the
data itself.

The APMC data model:

PIW maintains a high degree of data integrity within the dataset via
the APMC (Atomic Peer Moderated Collaboration) data model.  This is
very different from other collaborative data models such as the Wiki
(Wikipedia, et. al.) model in that there is no single authoritative
view of the data (with/without edit tree).  Instead each user/author
(Peer) has complete editorial control of data pertaining to a given
element (Atomic), allowing multiple user/authors to submit data for a
given element (Collaboration) which is then moderated by the
user/authors into a value hierarchy that affects display priority.
The highest values achieve display precedence while the lowest
values are not displayed by default.  In this case, the most
authoritative version of the data is the one that the most
users/authors (Peers) agree (Moderate) is the most accurate or offers
the most information.

This data model has advantages over the more traditional collaborative
Wiki model in that it easily prevents most forms of abuse.  Because
each user/author only creates/edits/deletes their own data and there
is no objectively authoritative data, it is impossible to hijack the
data (as Wikipedia is susceptible to).  It is also easy to remove [spam]
abusers because their data remains atomic and is not intermingled with
other user/author's data.  Although not currently in the server (for
security reasons), a method to remove all data submitted by a single
user would be easy to implement. 

Financial elements:

Income
--
August 8, 2004        Threshold Foundation      Threshold Grant         
10,000.00
--
Expenses
--
August 10, 2004         Permaculture Institute  Administrative fee      800.00
October 20, 2004        Chad Knepp              Programming             495.00
October 20, 2004        Chad Knepp              Reimburse Domain Name   14.96
January 7, 2005         Stephanie Gerson        Project Coordination    
1,000.00
July 26, 2005           Chad Knepp              Programming             630.00
January 18, 2006        Chad Knepp              Programming             765.00
January 18, 2006        Chad Knepp              5 Years Domain Name     76.80
--
Total Expenses                                                          
3,781.76
--
Current Balance January 27, 2006                                        
6,218.24

Based on expenses and the remaining balance, it may seem that little
has been done on this project.  However, an examination of
<http://dev.permaculture.info/> will clearly show that this is not the
case.  The fact that only 126 hours (3 weeks full time) is credited as
billable development time for the entire project so far indicates that
either we are a truly amazing devlopment team (not the case) or that
the majority of labor that has gone into PIW so far has been
voluntary.  All of the members of the PIW development team are
dedicated permaculturalists and care about the success of the project.
Although we do volunteer most of what we contribute, funding helps us
justify spending more of our *free* time creating something we believe
in.

Difficulties

Funding and commitment:

The PIW project has not achieved several of the self set goals we
created.  One possible reason for this is what we've termed the 10%
problem.  That being that we because we received approximately 10% of
our projected budget we made many compromises in order to stretch the
available funding, primarily by donating a large portion of our time
in order to get as much done with as little financial outlay as
possible.  Unfortunately for many of us, this relegated our commitment
to the project to one we participated in during our free time, giving
other commitments precedence.  While this approach has increased the
efficiency of our funding utilization, it has not helped us complete
aspects of the project within the timeframe we envisioned.  For
example, this report is approximately 5 months late and concludes that
we are approximately half done.

Several features are not yet implemented which include a moderation
system, [relationship] tagging, and a graph based interface for visual
browsing of interspecies relationships.

Version 0.3.1 contains a number of known bugs.  For instance there are
many cosmetic issues in the client, relating to inconsistent user
interface and layout.  Although xml-rpc is implemented in the server,
it is largely untested beyond proof of concept.  One currently known
issue is that some methods return boolean values which are unsupported
in the xml-rpc specification.  The dataset itself is based on the
Plants For A Future data schema, and, although it has been heavily
modified, it still contains some fields that are internally
inconsistent (habit, for example).  Many elements lack constraints,
although both the server and client layers add additional constraints
by filtering for valid input.


Future

Although our progress has been slow, the PIW project is alive and well.
Our current goals include user interface clean up and moving the
current development version <http://dev.permaculture.info> to the live
site <http://permaculture.info>.  At this point we intend to announce
our presence to the online permaculture community and actively solicit
participation in increasing quality and quantity of our dataset.
Further development will continue with the addition of a moderation
system, image support, tagging features including rudimentary browsing
methods, and locales (methods to differentiate data based on
climate/geographical differences).


Conclusion

The PIW project is continuing to develop, albeit slower than many would
prefer.  We still have funding and energy to continue, and we expect to
conclude this phase of development within 2006.  We feel that we have
achieved a major goal of creating a fully editable and extendable
dataset based on the Plants For A Future data.