I’ve been researching different search engines that can be used to index a limited set of websites. I’ve come up with these four as possible candidates:

ht://Dig does the job, but it has problems updating an index, and perhaps with database corruption. It will search pdf, word, and other documents if given proper parsers.

The Search Engine Project is a search engine that runs from php and indexes into MySQL (v4+).

SiteSearch uses Lucene (java based, from Apache Jakarta) as the indexer, has a php front end, and provides search and click statistics.

Spindle also uses Lucene. I don’t know much more than that.