Skip to Content.
Sympa Menu

sm-discuss - Re: [SM-Discuss] RFC: Prototype for full-text search indexing

sm-discuss AT lists.ibiblio.org

Subject: Public SourceMage Discussion List

List archive

Chronological Thread  
  • From: Ismael Luceno <ismael.luceno AT gmail.com>
  • To: sm-discuss <sm-discuss AT lists.ibiblio.org>
  • Subject: Re: [SM-Discuss] RFC: Prototype for full-text search indexing
  • Date: Mon, 8 Dec 2014 12:57:12 -0300

On Tue, 25 Nov 2014 02:29:23 -0300
Ismael Luceno <ismael.luceno AT gmail.com> wrote:
> It gives a little improvement over using grep, while still being
> fairly simple, using only plain text, and implemented purely in AWK.
>
> This prototype just returns the highest ranking results, and doesn't
> do any partial or fuzzy matching, nor case-insensitive matches, but
> the idea is to have at least partial matching and regular expressions
> working (both should be relatively easy to implement).
>
> Very early stage, lots of improvements coming soon.
>
> Please give it a look.
>
> Usage example:
> # Make an index:
> ; find /var/lib/sorcery/codex/test -name DETAILS -exec cat '{}' + |
> awk -f parse-spell | awk -f index > myindex
> # Search terms in the index:
> ; awk -f search myindex term1 term2 ...

New code at https://github.com/ismaell/sorcery-fts

Relevant changes:

* Now returns all matches, sorted by ranking.
* Ignores stop words.
* Parsing improvements.
* Now parses KEYWORDS too.

To do:
* Pre-processing to remove/convert symbols
* Ignore single characters
* Ignore numbers
* Stemming

Attachment: pgplmPLFL8zLE.pgp
Description: OpenPGP digital signature



  • Re: [SM-Discuss] RFC: Prototype for full-text search indexing, Ismael Luceno, 12/08/2014

Archive powered by MHonArc 2.6.24.

Top of Page