[b-hebrew] U-DOP was Generative Grammar

James Christian jc.bhebrew at googlemail.com
Thu Jun 24 10:15:24 EDT 2010

Hi Eric,

I don't know if you noticed but the U-DOT paper was specifically about
translation between the English-German language pair. German is an inflected
language. There is a very specific reason why this language was chosen. The
state of the art in machine translation has been statistical machine
translation for some time now. SMT works well on language pairs with similar
word order (e.g. the English-Italian language pair) but performs less well
on the English-German language pair because of long range dependencies.
U-DOT and a similar approach which proceeded it (synchronous CFG's) were
designed to tackle this kind problem and push technology frontier even

The main problem with these two methods is not so much the methodology as it
is the lack of suitable quantities of training data. This problem becomes
even more so evident with Biblical Hebrew as we have so little data to train

The real problems with U-DOT and SCFG's is that they are not accessible to
linguists. That is to say that while they create for themselves a complex
implicit rule set more complex and better defined than any human attempt
linguists are alienated from these approaches as there is no explicit rule
set to interact with. It may be interesting if the extracted rule set were
made explicit in some way such that theoretical linguists could interact
with the rule set and identify bugs and be able to suggest improvements.

James Christian

On 24 June 2010 05:25, Eric Inman <eric-inman at comcast.net> wrote:

>  Hi Jim,
> I read the papers and found them interesting.
> I think the issue of non contiguous phrases is related to but different
> from the issue of relative freedom of word order. Even though these papers
> made it a point to address non contiguous phrases, I feel the techniques
> that were used would still need to be modified if they were to be used on
> (relatively) free word order languages.
> To put it another way, a number of grammars and techniques are based on
> word order and the part of speech of each word. In inflected languages
> (which I'm assuming here to have greater freedom of word order), however,
> some of the information that is provided by word order in non-inflected
> languages is instead provided by inflectional affixes. Therefore grammars
> and techniques that are based only on word order and part of speech are
> working with less information when it comes to inflected languages. It would
> seem that they would need also to be based on inflectional affixes in order
> to be as effective with inflected languages as they are with non-inflected
> ones.
> I found the method of unsupervised learning for parsing and of determining
> the most probable parse trees to be very interesting. In general I think we
> need more approaches like this to help increase the level of objectivity of
> conclusions.
> So, with respect to Classical Hebrew, I think the U-DOP and U-DOT
> approaches would need to be adapted somehow to look at the inflectional
> affixes as well as the parts of speech and word order.
> Eric Inman
>  ------------------------------
> *From:* James Christian [mailto:jc.bhebrew at googlemail.com]
> *Sent:* Tuesday, June 22, 2010 6:08 AM
> *To:* Eric Inman
> *Cc:* dwashbur at nyx.net; b-hebrew
> *Subject:* U-DOP was Generative Grammar
> Hi Eric,
> forgetting about the exercise some of your comments made me feel you may be
> interested in reading the following papers:
> http://staff.science.uva.nl/~rens/conll06.pdf
> http://staff.science.uva.nl/~rens/mtsummit2007.pdf
> It is clear, as you hinted on in earlier posts, that a number of linguistic
> phenomenon can be best described by non contiguous phrases. Standard
> attempts at defining CFGs don't even tend to acknowledge the existence of
> these kind of linguistic phenomenon. Anyway, have a read of those papers. I
> get the feeling they might be more to your liking. They certainly are to
> mine.
> Obviously, there is no simple way that comes to mind of organising a simple
> exercise for list members to participate in. It would certainly be
> interesting to extend the U-DOP and U-DOT approaches to Classical Hebrew but
> doing so requires solid programming skills and a lot of time and patience.
> Let me know what you think about those papers. I would be interested in
> hearing your comments.
> James Christian
> On 21 June 2010 00:51, Eric Inman <eric-inman at comcast.net> wrote:
>>  Hi James,
>> I don't have any alternative suggestions as to what technology to use for
>> the type of exercise you're trying to set up. I don't think the issues I
>> raised earlier would prevent what you're doing from being a valuable
>> learning exercise.
>> Eric
>>  ------------------------------
>>  *From:* James Christian [mailto:jc.bhebrew at googlemail.com]
>> *Sent:* Sunday, June 20, 2010 1:36 PM
>> *To:* Eric Inman
>> *Cc:* dwashbur at nyx.net; b-hebrew
>> *Subject:* Re: [b-hebrew] Generation grammar and b-hebrew
>>   Hi Eric,
>> as I explained to you off list this isn't my approach. It was intended as
>> a simple exercise to inspire list members to get their hands dirty with
>> turning theoretical knowledge into something practical. Prolog (a technology
>> I would never use for a serious system) is an ideal starting point because
>> of its built in DCG handling which means no programming skills needed to get
>> started playing with a toy grammar.
>> Now, as far as I know the leading approaches at the moment involve using
>> either the LFG or the HPSG formalisms and there are quite large projects in
>> both technologies which are attempting to define wide coverage grammars of
>> modern languages. For a system driven by LFG see:
>> http://decentius.aksis.uib.no/logon/xle.xml
>>  <http://decentius.aksis.uib.no/logon/xle.xml>For information on systems
>> using HPSG see:
>> http://www.delph-in.net/
>>  <http://www.delph-in.net/>Now, I'm not sure what you mean by using an
>> approach that has been adapted for languages with a high degree of freedom.
>> If you have any suggestions of a technology we could use which is friendly
>> to list members who may have no computational linguistics experience or
>> programming skills then I'm open to suggestions. The two technologies
>> mentioned above for HPSG and LFG imply a steep learning curve and XLE is not
>> freely available anyway.
>> James Christian
>> On 20 June 2010 14:38, Eric Inman <eric-inman at comcast.net> wrote:
>>> Hi Jim,
>>> When referring to free word order languages, no one is using the term
>>> "free"
>>> in an absolute sense but rather in a relative sense. In addition, no one
>>> is
>>> using the word "free" to indicate that there are no rules or contraints
>>> governing the word order. From my review of your comments along with a
>>> quick, cursory review of what I could find by googling, my conclusion for
>>> the time being is that it would be a good idea to use a an approach that
>>> has
>>> been adapted for languages with a relatively high degree of freedom in
>>> word
>>> order.
>>> Your approach may very well prove productive if you carry it out, I just
>>> think there might be more efficient ways of proceeding. If you do proceed
>>> with this, I'll be interested in seeing what the results are.
>>> Eric
>>> -----Original Message-----
>>> From: b-hebrew-bounces at lists.ibiblio.org
>>> [mailto:b-hebrew-bounces at lists.ibiblio.org] On Behalf Of James Christian
>>> Sent: Sunday, June 20, 2010 2:48 AM
>>> To: dwashbur at nyx.net
>>> Cc: b-hebrew
>>> Subject: Re: [b-hebrew] Generation grammar and b-hebrew
>>>  Hi Eric,
>>> in my honest opinion there is no such thing as a free word order
>>> language.
>>> Yes. It is true that when we compare English word order with that of
>>> Koine
>>> Greek then the Koine Greek does exhibit more 'freedom' in its word order.
>>> But just what exactly do we mean by 'freedom'. Were the composers of
>>> Koine
>>> Greek documents just randomly throwing out declined noun phrases? Hardly!
>>> There are clearly patterns which are more dominant and patterns which are
>>> less dominant and guided decisions were made when these documents were
>>> generated. We even see similar phenomenon in English, a language we
>>> consider
>>> to not have free word order. Consider the following:
>>> 1) Mary went to school
>>> 2) To school went Mary
>>> 3) To school did Mary go
>>> 4) To school Mary went
>>> Sentence 1 is clearly the variant we would most likely encounter and yet
>>> there are contexts were 2, 3 and 4 could be naturally produced. However,
>>> this is certainly not a random pattern. There are clear reasons described
>>> by
>>> patterns which dictate when form 4 would be preferred over form 1.
>>> In any case, I cannot emphasize to you enough. Don't worry about the
>>> number
>>> of rules. This is not necessarily a bad thing. What you should be
>>> worrying
>>> about is whether your rule set over or under generates. Once you've got a
>>> rule set which doesn't over or under generate *then* is the time to start
>>> worrying if you could achieve the same thing with a more compact
>>> representation. However, I promise you this. If you give this experiment
>>> a
>>> go you will soon see that being in such an ground breaking position is
>>> far
>>> greater a problem than it may seem.
>>> I would rather have 1 million well defined rules that neither over nor
>>> under
>>> generate than 1 all embracing rule that produces every conceivable
>>> permutation of words both acceptable and unacceptable.
>>> James Christian
>>>  _______________________________________________
>>> b-hebrew mailing list
>>> b-hebrew at lists.ibiblio.org
>>> http://lists.ibiblio.org/mailman/listinfo/b-hebrew

More information about the b-hebrew mailing list