From: "JAMES CHRISTIAN READ" <JCR128 AT student.anglia.ac.uk>
To: b-hebrew AT lists.ibiblio.org
Subject: [b-hebrew] Karl's lexicon
Date: Sat, 25 Aug 2007 21:27:58 +0100
KWR: I guesstimate that the resultant file
would be from five to ten times as large.
JCR: How many entire does your dictionary have. A quick
electronic count of single words found in the xml Aleppo
Codex revealed that there are 43757 different words in
it.
Doing the same query for words that occur more than once
returns 19889 unique items. The same query for words
that occur more than twice returns 12910 items. More
than 3 times returns 9628. > 4X = 7657, >5 = 6412,
> 10X = 3572, > 20X = 1898. Research on large volumes
of English corpora have revealed that the English
language has a core vocabulary of roughly 2000 words
which make up for 75% of spoken and written English.
While the task of creating entries for every single
form could be daunting and possibly bear little fruit
in terms of words that occur once there is clearly a
core vocabulary of only 1898 words that could greatly
value from dedicated entries.
Applying a similar count approach to bi-grams only
1566 occur more than 10 times. Only 940 tri-grams occur more than 5 times.
Only 820 4-grams occur more than 3
times and only 866 5-grams occur more than twice.
If we were to decide on a cut off point of minimum
frequency 20 to qualify for entry into your dicionary
then the results would be: