discussion of the Creative Commons Metadata work

[bkhl AT stp.ling.uu.se (Björn Lindström)]

KirbyMeister <kirbymeister AT gmail.com> writes:

> Can the tokenization, segmentation, and linguistic annotation be
> automated, or does some of that need to be done by human hands?

It can, but how appropriate that is depends on the purpose of the
corpus. One common usage for corpora is to train programs that in turn
do these things with other text. For those usages, the annotation has to
be accurate, which means it has to be done manually. There are tools
that greatly simplify this, by letting a human interactively correct
annotation performed by a computer program.

On the other hand, if the purpose is to examine a particular area of
language, some degree of error can in some cases be acceptable, and the
human part of the process be unnecessary.

In this particular case, I'm mainly hoping to build a corpus of the
former type (since that would be the point of basing it on CC-licensed
material), so the annotation can probably not be completely automated.

> OT: Bjorn, you want a GMail account? I got 30 invites and I'd like to
> do something better with them then give them off to the Spooler :)

No, thanks. I'm so happy with Gnus.

-- 
Björn Lindström <bkhl AT stp.ling.uu.se>
Student of computational linguistics, Uppsala University, Sweden