full-text search on metadata/data

[livenson]

Implement a search engine that would index stored metadata/content.

Candidates:

• Xapian + python bindings (eg. xappy)
• Lucene

[deemoowoor]

I'd also suggest sphinxsearch. It's fast, efficient, fairly simple. Downside: it's designed for SQL DBs like MySQL and PostgreSQL. There IS a way to use it with arbitrary text, but it's not very robust IMHO. http://sphinxsearch.com/

[livenson]

Thanks. Seems like a nice candidate.

By any chance you feel like contributing a bit of code? :)

[deemoowoor]

I'll see what I can do. ;)

Lucene works with CouchDB with this fairly popular project BTW: https://github.com/rnewson/couchdb-lucene

Also there is this StackOverflow question w/ a good answer: http://stackoverflow.com/questions/5285787/full-text-search-in-couchdb ;)

Unless there are metadata properties which contain a lot of plain text to be full-text searchable, it might be more useful to just implement the search with CouchDB map-reduce API. After all, it's CouchDB, so it's supposed to be highly scalable? Still, pre-indexing even single-word properties may be a better idea sometimes...