csegurus.org down for a while (few days), oops

Due to a server move, the csegurus.org site was down for a few days. In short, I forgot to bring it back up after the server was moved. Oops.

Firefox Search Everywhere Toolbar is now public!

A while ago I posted information about my Search Everywhere extension for the Firefox browser. This extension is now public, and you can install it from the Firefox add-ons site. If you have already installed the Search Everywhere add-on from my site, consider reinstalling it from the official site. If you install from the official site, you will get automatic notification of upgrades.

New domain name live for new CSE website

My new unofficial Linked CSE (well, eventually CSE in general) site is now available under it's intended name: http://csegurus.org.

New CSE Website

I just finished the first draft of my personal Custom Search Engine website at http://cse.komarix.org. It's a bit rough, and missing a lot of content. Also, the domain name should change soon, to http://csegurus.org. Regardless, it's time I let it loose on the world. Let me know what you think!

Perhaps the most useful part of the new site is my unofficial Linked CSE tutorial.

More details for the Search Everywhere firefox extension

Once you have installed the Search Everywhere firefox extension (see my previous post) and played with it a bit, you might wonder what Exact, Path, Host, filter, and boost exact are for.

Link Extraction Parameters

Exact:

Use extracted URLs without modification, to create a very "narrow" search engine.

Paths:

Use the host+path from an extracted url, but omit any filenames.

Hosts:

Use only the host from extracted urls to create a "broad" search engine.

Extraction Modifiers

Filter:

Some of the extracted URLs might be dominate search results for almost any query. For example, if we extract http://nih.gov, it is unlikely any other extracted URL will appear in search results. By enabling this option, the search engine will avoid certain overly-dominant URL patterns, like nih.gov/*.

Boost exact:

This allows combination of host or path URL extraction, with an extra layer of exact URL extraction. Exact urls will be prefered in the search results.

Other Options

New Tab

Open search results in a new tab.

More features are coming! Adding more features will require interface improvements, too, so let me know sooner than later what you like and don't like.

New Firefox Extension: Search Everywhere

I recently created a Firefox extension to show-off a new type of Google Custom Search Engine: the Linked CSE. The extension is called Search Everywhere, and it lets you search over the content linked from your web browser's current page. Here's where it might be useful: suppose you are hanging out at the Dapper Rat website, and become interested in cages for pet rats. The Dapper Rat has a terrific page of nearly 300 links pointing to reliable pet rat information. You need a search engine that searches over this reliable information. That is exactly what the Search Everywhere toolbar does, using a Linked CSE.

After installing the Search Everywhere extension, visit the page of links and open the toolbar (by clicking on the icon in your status bar). Type "cage" in the input box, hit search, and you'll get these rat cage results. Did you notice that the actor Nicholas Cage did not show up? Nor did composer John Cage or heavy metal band Cage, and there wasn't a single reference to cage fighting (compare to this regular google search for [cage]). And if you trust the Dapper Rat website, you might trust these results more than regular Google results for the query [rat cage].

How does it work? A Linked CSE is defined by an url that returns an XML CSE definition. The URL can point to a CGI script, for instance a script that scrapes links from a webpage and outputs XML. Once you have an URL that defines a CSE, you can search for STUFF using this url: http://www.google.com/cse?cref=URL&q=STUFF. Note that you need to url-escape both URL and STUFF, for example using the Javascript encodeUriComponent() function. The Search Everywhere toolbar does all this for you, using an url that looks something like this:

  http://www.google.com/cse?cref=SCRAPER(window.location())&q=QUERY

Note that window.location() is just Javascript that means "the webpage you are viewing". The Search Everywhere toolbar uses a CGI script at http://www.google.com/cse/tools/makecse to perform the scraping, and QUERY is whatever is typed into the toolbar's search box. After escaping the cref and q arguments, the complete url (broken across a few lines, for readability) is

  http://google.com/cse?
      cref=google.com%2Fcse%2Ftools%2Fmakecse%3F
          url%3Ddapper.com.au%252Flinks.htm
      &q=cage

Notice that dapper.com.au/links.htm is the url= argument for makecse, which is the argument for cref=, and hence the "/" in dapper.com.au/links.htm is double-escaped to become %252F.

Please let me know what you think about this extension. Note that you have to restart Firefox after installing it, and after that you should see the "G-and-link" icon in the status bar at the bottom of your browser window. For now, you can get Search Everywhere from my website, but eventually it should be available from the Mozilla Add-ons website.