Public SourceMage Discussion List

[Ismael Luceno <ismael.luceno AT gmail.com>]

On Tue, 25 Nov 2014 02:29:23 -0300
Ismael Luceno <ismael.luceno AT gmail.com> wrote:
> It gives a little improvement over using grep, while still being
> fairly simple, using only plain text, and implemented purely in AWK.
> 
> This prototype just returns the highest ranking results, and doesn't
> do any partial or fuzzy matching, nor case-insensitive matches, but
> the idea is to have at least partial matching and regular expressions
> working (both should be relatively easy to implement).
> 
> Very early stage, lots of improvements coming soon.
> 
> Please give it a look.
> 
> Usage example:
> # Make an index:
> ; find /var/lib/sorcery/codex/test -name DETAILS -exec cat '{}' + |
>   awk -f parse-spell | awk -f index > myindex
> # Search terms in the index:
> ; awk -f search myindex term1 term2 ...

New code at https://github.com/ismaell/sorcery-fts

Relevant changes:

* Now returns all matches, sorted by ranking.
* Ignores stop words.
* Parsing improvements.
* Now parses KEYWORDS too.

To do:
* Pre-processing to remove/convert symbols
* Ignore single characters
* Ignore numbers
* Stemming

Attachment: pgplmPLFL8zLE.pgp
Description: OpenPGP digital signature