Search result aggregation is not a new thing. The MetaCrawler search engine has been doing this sort of thing for years. Unfortunately, the search information is bounded and limited to just a couple of pre-determined set of engines.

OpenSearch is a set of standards which opens up search aggregation. Any search engine can share their search results using the OpenSearch standards, so that search aggregators can pull it all together. All the OpenSearch information and documentation is actually pretty well written (unlike other specifications which are dozens of pages long and too difficult to get through) and are available at www.OpenSearch.org.

One example is that blog searches can be aggregated. I operate a few different blogs, each one with its own search function. If I were to want to look for the word “oregon” in my blogs, I would have to visit each one individually and use their individual search engines. I could try using a third party search engine limited to the scope of only my blogs, but if that search engine hasn’t visited all the posts in all my blogs, then I’m not getting great search results back. Now suppose that each of my blogs’ search functions supported OpenSearch. Then I could simply use a search aggregator and search all of them at once, and all their results displayed all at once. Bingo.

OpenSearch version 1.1 uses standard technology and is composed of two main parts. There is the OpenSearch description file which tells an aggregator or search application how to perform a search. This is what Firefox 2 and IE7 use for their search plugin format. The description file is XML with a fairly simple vocabulary. The second part is the search results, which uses RSS and Atom format standards along with a few custom tags to return the search data. Using RSS and Atom makes it easier to write search clients and servers which use OpenSearch because there are already libraries for the standards and developers are already familiar with the technology.

So when you use a search aggregator or application, such as A9.com, the application uses the description files to properly format search queries. The queries are sent to the search servers, which return the search results as an Atom or RSS formated file. The application then parses and displays in some meaningful way the search results from the various sources.

In the past we were limited to using either one generic monstrous search engine like Google, or bouncing around to many different tiny little specific search engines. With OpenSearch, we should be able to instead search many specific search engines without having to bounce around.

Thank you all very much for reading. I appreciate every visitor, including the rude ones, but not really the ones that spam.

For work, I’ve been working on a new search engine called LDSsearch.com. It is finally to the point where I feel like I can start showing it off and get feedback on it.

This search engine only indexes Internet content which has been identified to be LDS-friendly material. The idea is to help members, and investigators to find accurate information about the mormon religion without having to sort through anti-mormon sites.

LDSsearch.com uses the open-source search engine, Nutch. Nutch made it faster and easier to set up a search engine. We’re using version 0.8-dev, which is in need of some very good documentation.

Please take a look at it, and tell me what you think by commenting on this post.

ht://Dig does the job, but it has problems updating an index, and perhaps with database corruption. It will search pdf, word, and other documents if given proper parsers.

The Search Engine Project is a search engine that runs from php and indexes into MySQL (v4+).

SiteSearch uses Lucene (java based, from Apache Jakarta) as the indexer, has a php front end, and provides search and click statistics.

Spindle also uses Lucene. I don’t know much more than that.