Permaculture Information Web

[Chad Knepp <pyg AT galatea.org>]

Hi all,

Here is longish response to Rich M., Heide, John S. and Bear about
relationships.  The short version is that we need to talk more about
this and hopefully here is some stuff to work off of.  I find working
with actual examples very helpful in explaining and understanding the
problem.  If you can, please use examples of relationships.

Richard Morris writes:
 > Chad Knepp wrote:
 > 
 > > Need some help here,
 > > 
 > > Although we have talked about it before, I'm still sort of unclear on
 > > what relationships are to the extent that I'm not coming up with an
 > > implementation.  Can folks suggest some stuff?  Specifically I think
 > > it would help me to hear some fantasies as to how they might work; as
 > > in you define a relationship this way with these elements and when you
 > > search for this or are looking at that you get a path to the
 > > relationship.
 > > 
 > > Here are some questions that might help.
 > > 
 > >   o Since we only have plants at the moment, how should we represent
 > >     relationships with non-plant elements (animals)?
 > > 
 > This is a sort of name space issue, i.e. how do we refer
 > to objects.
 > 
 > One option is to give each object a class.
 > Typical classes could be Plant, Animal, Location, Habitat,
 > Guild,User. Then we could refer to an object as
 > Plant:Salix_alba or Animal:Bumble_Bee.
 > In a URL this could be http://permaculture.info/plants.py?Salix+alba
 > or
 > http://permaculture.info/search.py?class=plant&name=Salix+alba
 > Internally we don't need to store the full url, just the
 > class and the object name. (better as it allows us to
 > change the exact structure of a url later).
 > 
 > The alternative is a wiki style where everything is in the same namespace.
 > http://permaculture.info/search.py?Salix+alba
 > http://permaculture.info/search.py?Bumble+Bee

This is pretty much exactly how things are already.  Some of the minor
differences from your description is that I give use numbers for
object ids not real word names.  The actual xml (kfml) for doing links
like above is: <link plant="6047">Salix Alba</link>.  The html client
(Html.py) converts this into <a
href="http://permaculture.info/cgi-bin/eden?plant=6047";>Salix
Alba</a>.  You can even embed searches with things like <link
search="Salix alba">Look for white willow</link>.  Current objects include
plants, authors, comments, uses, references, and [comming soon]
relationships.

 > >   o What sorts of information should a relationship have?  Things
 > >     like: A list of plants contained in the relationship.  One [or more]
 > >     word[s] that describe[s] the relationship.
 > 
 > Main questions is a relationship one directional like a
 > <a href=...>
 > or bi birectional
 > <link source=... dest=...>
 > 
 > Also worth considering if is a 'one to one'
 > or 'one to many' relationship. I'd think we should just have 1 to 1 
 > relationships.
 >
 > If you want to have one to many relationships
 > then use an intermediaty object.
 > A typical example of a one to many relationship might be a guild which 
 > has lots of plants.
 > Each plant could refer to the guild (one relationship per plant)
 > and the guild, a seperate object in its own right.
 > <link source="Plant:Salix+alba" dest="Guild:Willow+Guild">
 > <link source="Plant:Allumn+cepa" dest="Guild:Willow+Guild">

Well, even if the relationship is one element relating to another
element, in terms of table structure, I don't think we can have any
unique columns.  For example:

object  |  relationship  | object
---------------------------------
plant A   guild member     Plant A guild
plant A   guild member     Plant B guild
plant A   guild member     Plant C guild
plant B   guild member     Plant B guild
plant C   guild member     Plant C guild
plant D   guild member     Plant B guild

... is really a many to many table.  Although each row is unique,
every column has duplicates.

How about this?  No better, but lighter.

object  |  relationship
-----------------------
plant A   Plant A guild member 
plant A   Plant B guild member 
plant A   Plant C guild member 
plant B   Plant B guild member 
plant C   Plant C guild member 
plant D   Plant B guild member 

This is close to del.icio.us style loose tagging.  I kind of like this
from a weight standpoint.

 > Do do think its wise to allow relationships to have a type
 > and also posibly a description.
 > <link source="Plant:Salix+alba" dest="Guild:Willow+Guild"
 >      type="Guild Member" descript="main plant in guild">

Well, this adds weight to the object.  There are several different
ways we can approach relationship implementation, from heavy to light.
I was favoring a light weight approach because I suspect it would be
more flexible in the long run (and we have so many to add).  Heavier
objects could include lists which is what I think you're suggesting.
It would be easy enough to allow attaching a comment to the
relationship that I don't think relationships need built in
descriptions.  This keeps it light and gets at what I think you
wanted.

 > >   o Should relationships be owned? editable? moderateable?
 > 
 > Assuming that the relationship was part of a comment. Then it would
 > have the have the same owership and permisions as the comment. 
 > 
 > In the past I've played with having everything with an owner.

I was thinking they would be completely separate objects.  By not
being owned, relationships could be submitted but not edited or
deleted.  They could be moderated with or with out ownership, but
moderation adds a lot of weight... 

 > >   o What's the difference between a quality/attribute and a
 > >     relationship?  For example is nitrogen fixing a relationship or an
 > >     attribute?  What about alleopathic (sp?) properties?
 > 
 > I'd say nitrogen fixing is an attribute, but
 > habitat types, plant uses are relationships.
 > I'd be very inclined to have Habitats and Uses as top level classes.
 > So its posible to have a page per habitat describing the habitat
 > and listing the plants in that habitat.
 > 
 > >   o What's the difference between a comment and a relationship?
 > 
 > Or even, plant, comment, relationship.

Wait, you're not supposed to add more complexity to my question,
you're supposed to answer it;-)

 > >   o How is a relationship different from a natural plant community?
 > 
 > Links and nodes. In a graph theory way of looking at it
 > a natural plant community is a 'node' and relationships are 'links' to
 > and from that node.
 > 
 > > Also the techie folks on the list can feel free to chime in with
 > > suggested schema (relational/SQL stuff).  Remember different
 > > relationships are probably going to be one-to-one, one-to-many,
 > > many-to-one, and many-to-many.
 > 
 > Bigee, could have one tabel per class, or one big table with a class field.

By class do you mean relationship?  The current DB is pretty much one
table per object type with some exceptions to achieve a higher level
of normalization.   

 > Relationships are probably best in a separate table (similar
 > to 'edible use details' but with a couple of other fields.
 > 
 > In the past I've have a concept of a contribution, similar
 > to your comment idea. Everything in the database
 > had the following fields
 > 
 >      `Contributer` TINYTEXT NOT NULL,
 >              # "Who submitted this record","Must corespond to an entry in 
 > the users 
 > table",NULL
 >      `ContribDate` TIMESTAMP,
 >              # "When it was submitted","Entering NULL will insert the 
 > current 
 > date",NULL
 >      `ContribAction` Enum("Req Add","Req Update","Req Del") NULL,
 >              # NULL,"What to do with the record, i.e. instruction for the 
 > editors
 >              # Add means just add some info
 >              # Update means a change in info
 >              # Delete indicates that some info might be incorect","Might 
 > at some 
 > later date want to change the ENUM for this"
 >      `ContribNotes` TEXT,
 >              # "Notes supporting this contribution",NULL,NULL
 >      `ContribRefs` TEXT
 >              # "references to support this contribution",
 >              # "a comma seperated list of reference codes in the 
 > references table",NULL

This would be a fairly heavy weight implementation, and looks like it
would require some interaction.  

Heide Hermary writes:
 > 
 > 
 > Chad Knepp wrote:
 > 
 > >  o Since we only have plants at the moment, how should we represent
 > >    relationships with non-plant elements (animals)?
 > >  
 > >
 > There are not just animals, but microbes as well, and they can be 
 > antagonistic (as in diseases, pests, etc.), or synergistic, as in 
 > mycorrhizae, nitrogen fixing bacteria, pollinators, etc..  Knowing the 
 > specific species involved in all relationships would be helpful, 
 > wherever that is possible.  For instance not all nitrogen fixing plants 
 > form relationships with the same species of basteria. Most plant 
 > diseases and pest are very finnicky about their diet (i.e. the plants 
 > they care to eat).  I think if we want to go beyond current concepts we 
 > can start right here, by treating all of these as equals and  partners 
 > to their relationships

On a philosophical level I agree with you, but I think I was looking
for something more technical.  This is still an unanswered question, I
want to resubmit.  Restated: How do handle relationships to categories
that aren't currently in the database.  Categories that we may someday
add and ones that we have no intention of adding in the future.  Do we
want to include microbes as a distict objects at some point for
example?... and how does this affect our definition of relationships.
If we can say that "plant 6047" has a relationship of "is food for"
with "a beaver" but don't have beavers yet... see the problem?

 > >  o What sorts of information should a relationship have?  Things
 > >    like: A list of plants contained in the relationship.  One [or more]
 > >    word[s] that describe[s] the relationship.
 > >  
 > >
 > See above. So much more information is needed
 > 
 > >  o Should relationships be owned? editable? moderateable?
 > >  
 > >
 > Considering how little information we currently have about these, people 
 > should not be restricted from entering information.  I am questioning 
 > the need to "own" a relationship between organisms.  I see this more 
 > like a huge multidimensional web, where different organisms have many 
 > different relationships with many other organisms. Thus specific 
 > relationships don't need to be defined per se, they either are or aren't.

I totally agree with the importance of unrestricted entering of
information!  Owned in this context has a technical meaning that
doesn't actually have a quality of exclusive possession.  

 > For instance, a plant can have many pollinators. Would you define the 
 > relationship as "pollination", or as a relationship between 2 
 > organisms.  In case of the latter, the relationship can actually be 
 > multi-dimensional (such as a plant providing nectar to ladybugs that 
 > come and feed on aphids, or whatever).
 > 
 >  I don't know if that makes sense.

I think this is a great example of the complexity of relationships.
Here is a tabled representation:

object  |  relationship  | object
---------------------------------
ladybug   pollinator       plant A
plant A   food             ladybug adult
plant A   food             aphid
aphid     food             ladybug larva

Does this do it?  In this case what is most noticable to me is that
"food" seems like a gross over-simplification.  Maybe "pest" in some
cases?  Ooohhh!  There is an additional relationship implied
(possibly) but not described here.  The ladybug eating the aphids
helps the plant.  How can you describe that with two objects and a
verb?  

 > >  o What's the difference between a quality/attribute and a
 > >    relationship?  For example is nitrogen fixing a relationship or an
 > >    attribute?  What about alleopathic (sp?) properties?
 > >  
 > >
 > I already talked about the nitrogen fixing relationships.  Allelopathic 
 > compounds are products produced by plants, not necessarily attributes 
 > (unless we want to take the human centered perspective here).  As such 
 > they are used to regulate the plant's relationships (i.e. protect itself 
 > from predators, competitors, etc., BUT they may also and simultaneously 
 > be used to encourage other organisms! -). Who knows if we can ever 
 > understand all that, it is sooo complex, but it would be useful for the 
 > database to NOT limit the evolution of our understanding.

Very tricky.  Not only is our understanding human biased, but we are
using specific computer technologies that further shape our
information experience.  Sometimes I fear that the attempt of modeling
relationships itself will force us into a way of
thinking/understanding that might limit more than help.  That's why
getting this right is so important.

 > >  o What's the difference between a comment and a relationship?
 > >  
 > >
 > A comment might define or describe the relationship.
 > 
 > >  
 > >  o How is a relationship different from a natural plant community?
 > >  
 > >
 > Relationships are specific arrangements between organisms, a natural 
 > plant community is only one manifestation.  In  reality those plants 
 > have relationships with all kinds of other organisms, including insects, 
 > animals, microbes.  How detailed do you want to get?

I also like Rich M's answer above.

John Schinnerer writes:
 > Aloha,
 > 
 > >>   o What's the difference between a quality/attribute and a
 > >>     relationship?  For example is nitrogen fixing a relationship or an
 > >>     attribute?  What about alleopathic (sp?) properties?
 > > 
 > > I'd say nitrogen fixing is an attribute, but
 > > habitat types, plant uses are relationships.
 > 
 > Then again, from the perspective of those little N-fixing bacteria, it 
 > might look more like a habitat type, or a plant use...
 > 
 > N-fixing at least for some legumes depends on the presence of the 
 > appropriately 'related' (ahem... ;-) bacteria in the soil - and vice 
 > versa...if a plant that has the *potential* to relate 
 > with/support/provide habitat for n-fixing bacteria is put where they 
 > aren't present, it won't have that 'attribute'...so I'd say it's not an 
 > attribute.
 > 
 > Attributes would *only* be things that have no significant 
 > interdependency on something external to the plant - an intrinsic 
 > property of the plant.
 > So most of the taxonomic stuff, if I understand it right, would be 
 > attributes (e.g. the descriptive taxonomies used to classify plants - 
 > 'clusters of three leaves', 'five-petaled orange flower', 'three evenly 
 > spaced branchings per stem', etc. etc.)
 > If the info depends on some relationship(s) then it's a relationship...!

What about soil alkalinity and other climate prefs.?  Are these
relationships, attributes, or something else entirely?

 > As for your more general question, I still think of this as a DB I would 
 > go to for finding plants based on a variety of permaculture-centric 
 > information - functions and relations and attributes in the PC sense.
 > 
 > Not that I need to do human-English searches - just some narrative 
 > examples of what I mean:
 > 
 > "I need a tree that provides fodder for goats, is happy in zone 'X', 
 > provides medium shade from a high leaf canopy, tolerates wet feet, and 
 > typically maxes out at 40-50 ft. high."
 > 
 > "I need a slow-growing bush that provides food and shelter for XYZ types 
 > of birds, nectar for bees, can be maintained as a hedge-row and 
 > tolerates drought."
 > 
 > "I need a nitrogen-fixing, low-growing ground cover that tolerates full 
 > sun to medium shade, survives occasional foot traffic and does not 
 > negatively affect/does not compete with the growth of flowers A or B."
 > 
 > ....and so on.

Also my goal and something partially implementation in/with Eden.
This doesn't really depend on an implementation of relationships
though.  This sort of info could be pulled out of comments just as
well. 

I think value of relationships is in non-text representation and
manipulation... the GBI stuff.

 > I don't have a clear articulation at the moment, but the one to one 
 > relationships thing seems off somehow...how would you answer questions 
 > like above based on just 1-1 relations stored in the DB?  Would that be 
 > an easy search to code, or a difficult/expensive one?

I don't think the coding part will be hard.  I'm mostly worried about
having millions of heavy weight objects and the load of searching
through them all.

Bear Kaufmann writes:
 > >>   o How is a relationship different from a natural plant community?
 > > Links and nodes. In a graph theory way of looking at it
 > > a natural plant community is a 'node' and relationships are 'links' to
 > > and from that node.
 > It seems if one viewed a plant community as a node you'd be looking at  
 > one system level up from the ecological relationships within that node.  
 > But yes, technically, you'd need to say this, this, and this are part  
 > of the guild/community.
 > 
 > >>   o Should relationships be owned? editable? moderateable?
 > >
 > > Assuming that the relationship was part of a comment. Then it would
 > > have the have the same owership and permisions as the comment.
 > >
 > > In the past I've played with having everything with an owner.
 > I'd also be inclined to try to have relationships, comments, etc be a  
 > subclass of one primary object type. Comments have some specific forms  
 > of relationships to objects and other comments (always a tree form, no  
 > network). The superclass has owner data, source?, date stamp, handles  
 > relationships, outputs the object for full text search table(?), etc.

Right, comment trees can be implemented by just having a parent.
Relationships would need to have multiple linkages or a functionally
equivalent structure.  Yes, relationships and comments seem to have a
lot in common, which is why I've been asking what the difference is.
Not really understanding your last sentence.

 > I don't know if this is exactly how it's setup now, or if this is  
 > something a later release.
 > 
 > Also, how are references handled right now? Does the ref list at the  
 > bottom get created on the fly or is in in the text-blob already? I'd be  
 > inclined to allow a source reference relationship at some point....
 > (People could enter the number/letter as they are now [32] into a  
 > field, plus a link to pop up a key to sources...)

Half of each.  About two years ago I wrote a perl spider to build an
index table of the plant_id, reference_id correspondence.  It
currently selects the references for a given plant id and then
[dynamically] builds the references part.  This only works for a
static dataset.  If new references are added there is not a
corresponding way to update the index table.  

Anyway, this is why I've been pushing the idea of XMLizing this part
of the data.  [32] would become [<link reference="32">32</link>] or
something.  Although it is a concern to mark up the data in ways that
may decrease its accessibility '[32]' is a "mark up" and a horribly
inaccessible one at the moment.  The other alternative of removing
them and externalizing the reference (in another table) does reduce
some of the information (the references often pertain to a particular
sentence in the text).  This is kind of offtopic, but I'm open to
talking about this more/again.

 > Something else to add to the pot...(not to overwhelm our programmer..  
 > :-) Let me know if I should be quiet Chad.. :-)
 > 
 > There's the concept of faceted searching, which seems like an  
 > interesting technique, which would be helpful in searching for guild  
 > members...
 > 
 > Basically....Look over all the objects in the DB, and find the range of  
 > properties that they have.
 > List the available search criteria from that range (n-fixer, bog, sun,  
 > shade, high ph, low ph, Asteraceae, etc, etc)....clicking one searches  
 > the DB against that criteria...
 > 
 > Look over the returned objects for the range of properties that they  
 > have...and list the criteria....
 > And on it goes...
 > 
 > The cool thing is it shows what's available and doesn't lead to dead  
 > ends...
 > If you choose n-fixer....suddenly, the families available drops way  
 > down...
 > If you choose bog, suddenly you find that sedges and rushes are  
 > there...look at the medicinal properties available from a bog...no  
 > fruit producers, but hey, corms! (semi-educated description here).
 > 
 > It's probably a good bit of work to implement, but seems a handy way to  
 > look at the data...
 > 
 > Here's an implementation of the concept:
 > http://www.siderean.com/delicious/ 
 > facetious.jsp?tn=0subject&tv=plant&ss=1

Interesting and not that different from ideas I've entertained myself,
except for the self-referencing aspect.  We could fake that though
because we are [so far] dealing with a small finite number of
different objects with static attribute types.

 > >>>   o What's the difference between a quality/attribute and a
 > >>>     relationship?  For example is nitrogen fixing a relationship or 
 > >>> an
 > >>>     attribute?  What about alleopathic (sp?) properties?
 > >> I'd say nitrogen fixing is an attribute, but
 > >> habitat types, plant uses are relationships.
 > >
 > > Then again, from the perspective of those little N-fixing bacteria, it 
 > > might look more like a habitat type, or a plant use...
 > >
 > > N-fixing at least for some legumes depends on the presence of the 
 > > appropriately 'related' (ahem... ;-) bacteria in the soil - and vice 
 > > versa...if a plant that has the *potential* to relate 
 > > with/support/provide habitat for n-fixing bacteria is put where they 
 > > aren't present, it won't have that 'attribute'...so I'd say it's not 
 > > an attribute.
 > Trifolium repens -> interaction[symbiotic] -> Rhizobium leguminosarum
 > Trifolium repens -> common_name[English] -> White Dutch Clover
 > Trifolium repens -> interaction[symbiotic] -> Rhizobium trifolii 
 > (source: http://www.rsnz.org/publish/nzjar/2002/017.php)
 > Rhizobium -> fixes[atmospheric] -> nitrogen
 > Rhizobium -> habitat -> aerobic
 > Rhizobium -> morphology[shape] -> rod
 > 
 > Esh, what a tangled web of relationships...inference engine anyone?
 > Probably want to put Legume between the clover and the Rhizobium (many 
 > -> legume, legume -> rhizo) But man, googling for data, there's been so 
 > much research done on all these features. It's a shame there already 
 > isn't a machine readable (RDF?) abstract form for this data.

As in my re: to Heide, inference is challenging with lightweight
implementations such as this.

 > > Attributes would *only* be things that have no significant 
 > > interdependency on something external to the plant - an intrinsic 
 > > property of the plant.
 > > So most of the taxonomic stuff, if I understand it right, would be 
 > > attributes (e.g. the descriptive taxonomies used to classify plants - 
 > > 'clusters of three leaves', 'five-petaled orange flower', 'three 
 > > evenly spaced branchings per stem', etc. etc.)
 > BTW, faceted searches seem like they'd work great for taxonomic 
 > identification...
 > Find a trait of the plant (that you understand or can observe), flower 
 > color = red...
 > OK, all the red flowers come up...
 > It's got a tree habit...
 > OK, getting close, only 30 matches...
 > Pea-shaped flower...
 > 2 matches. Locust...(racking my own limited DB... :-P )

This explains faceted searching better to me.  Yes this would be some
interesting coding.  How do you know what to start with?  Like what
start with flower color and not flower shape?  Or do you start with
all of the options and it just continues to narrow the resuts?...


Cheers,
Chad

-- 
Chad Knepp
python -c 'import base64;print base64.decodestring("cHlnQGdhbGF0ZWEub3Jn")'