Biblical Hebrew Forum

["JAMES CHRISTIAN READ" <JCR128 AT student.anglia.ac.uk>]

KWR:  I guesstimate that the resultant file
would be from five to ten times as large.

JCR: How many entire does your dictionary have. A quick 
electronic count of single words found in the xml Aleppo 
Codex revealed that there are 43757 different words in 
it.

Doing the same query for words that occur more than once 
returns 19889 unique items. The same query for words 
that occur more than twice returns 12910 items. More 
than 3 times returns 9628. > 4X = 7657, >5 = 6412,
> 10X = 3572, > 20X = 1898. Research on large volumes 
of English corpora have revealed that the English
language has a core vocabulary of roughly 2000 words 
which make up for 75% of spoken and written English. 
While the task of creating entries for every single 
form could be daunting and possibly bear little fruit 
in terms of words that occur once there is clearly a 
core vocabulary of only 1898 words that could greatly 
value from dedicated entries.

Applying a similar count approach to bi-grams only 
1566 occur more than 10 times. Only 940 tri-grams occur more than 5 times. 
Only 820 4-grams occur more than 3 
times and only 866 5-grams occur more than twice.

If we were to decide on a cut off point of minimum 
frequency 20 to qualify for entry into your dicionary 
then the results would be:

1-grams = 1,989
2-grams =   588
3-grams =    93
4-grams =    34
5-grams =    12

For a grand total of 2716.

Now is that really so much work involved to make 
reading over 75% of the Tanakh fluidly accessible to 
learners of all types?

James Christian Read - BSc Computer Science 
http://www.lamie.org/hebrew       -  thesis1: concept driven machine 
translation using the Aleppo codex 
http://www.lamie.org/lad-sim.doc  -  thesis2: language acquisition simulation
http://www.lamie.org/ngelta








































































-grams =   588
3-grams =    93
4-grams =    34
5-grams =    12

For a grand total of 2716.

Now is that really so much work involved to make
reading over 75% of the Tanakh fluidly accessible to
learners of all types?

James Christian Read - BSc Computer Science
http://www.lamie.org/hebrew       -  thesis1: concept driven machine 
translation using the Aleppo codex
http://www.lamie.org/lad-sim.doc  -  thesis2: language acquisition simulation