The Google Search Server

An anonymous reader submitted a reasonably indepth review of the Google search appliance. The guys from anandtech put it through it's paces, and included a variety of pictures and comments on one of those Google products most of us will probably never play with.

Comments

AnandTech not very search optimization saavy

[DeadSea]

The Mini considers any unique URL string to be a unique document, which makes sense (but is a bit surprising the first time that you run an index). After four hours of indexing, the Mini had managed to reach its document limit and we had to improvise.

Anybody who doesn't know that search engines consider each url to contain a unique document does't know much about getting their site to be properly represented in search engines.

Their solution was to create a list of urls for the appliance to crawl. If they had to do that for the search appliance, there is no way that googlebot, msnbot, or yahoo slurp is going to be able to properly index their site.

Your public accessable urls need to managed and canonicalized through judicious use of robots.txt, 302 redirects, site wide linking, and just plain thinking out the layout of your site.

Re:AnandTech not very search optimization saavy

[Anonymous Coward]

... which flows right into this statement:
A word to the wise: don't let the Mini crawl your entire site without keeping a close eye on it.

The same could be said of any search engine, or any automated process for that matter. We use ht://Dig and the issues are the same, except ht://Dig can be run locally on the server, saving bandwidth (and speeding up the indexing process) by indexing locally and re-writing urls for static files, through apache for dynamic, it's free, and you aren't limited to 100000 documents. It supports the same feature set, minus the Google Gui.

Of course, it does have a steeper learning curve... you actually need to understand how search, url filters, regex, synonyms, etc works.

I'd provide screenies, but most people glaze over when confronted with terminal output ; ) A shell just isn't as hip as an html gui. What else can I say?

L8,
AC

Re:Neat insides

[b0r1s]

These are neat little boxes - we've managed 2 (the yellow appliance, and the blue mini appliance), and the performance of both was pretty nice.

The tools google provides (very easy binary updates, strong web control panel, for example) turn the relatively common task into a dead-simple, point-and-click configuration.

They even provide a decent interface for skinning the search pages, and while it's not perfect, it's certainly adequate for even the best looking sites on the internet.

It's "its"!

[dtmos]

The guys from anandtech put it through it's paces

It's really easy: It's "his", hers", and "its". Even a flower knows!

--cycling through grammar Nazi mode. Please wait.

Re:It's "its"!

[Traa]

Use "it's" when you can replace it with "it is"

Well, that is what someone told me anyway. English is not my primary language, if the above is not correct then please don't shoot me.

Re:It's "its"!

[radishes]

and use its' when it's possesive

john's coming to get johns' hat

Don't listen to this guy. He has lied to you twice. 1) Its' is never valid. 2) The example with John is just so wrong it hurts. "John is coming to get John's hat." You use 's for possessive; s' is for possessive plural, like this: "Slashdotters tend to live in their parents' basement."

Re:where's the raid?

[slim]

I guess if you want RAID, you pay more than $3,000.

What you're really buying here is closed-source software, wrapped in the hardware that turns it into an "appliance". Assume $2,000 of that $3,000 pays for the software.

By specifying the hardware in this way, and by keeping the BIOS and root passwords to themselves, Google greatly simplify their support role.

This is common practice: an IBM HMC (Hardware Management Console) is a 1U PC with a custom Linux distribution and the management software preinstalled. You don't get the root password; you just use the software as delivered.

Re:GPl compliance

[Anonymous Coward]

http://code.google.com/mirror/gsa.html

For those who're interested...

[Homicide]

I admin a full blown Google Search Appliance, the mimi's big brother.

If you want the specs:
Dual Xeon 2.6GHz
12GB RAM
4 250GB HD's in RAID(something) with a hot-swap spare.

Never tried taking off the cover though, since we want to keep the warranty.

All of the money you pay is a license for the software on the box, the system itself is effectively free, so once the 2 year warranty expires, you've effectively got a nice powerful linux box for free. You can keep running the software, but without any support.

As for performance, this thing works great, we have about 250,000 pages that it can index, both public and private (and it can do searches cleverly checknig username/pasword to see if you should have access to certain results), and we've had nothing but positive responses from our users. The results come up quickly, they're the results people want, and the results that management think should be at the top, are at the top.

Re:For those who're interested...

[Homicide]

It submits a HTTP HEAD request for the URL to the server the page is on, with the username and password supplied, so the server at the other end decides if you should be able to see the search results, thus saving you from having to faff around telling the google box who can get to what pages.

product review: the yellow GSA

[msblack]

We evaluated on of those yellow Google search appliances (GSA) and experienced very mixed results. The appliance is very easy to set-up and launch an initial scan of our website.

The GSA will blindly search all web servers in your domain. When setting-up the GSA, you give it an initial page from which to start crawling and baseline domains. For example:

Inital page: http://www.slashdot.org/
Domain(s): .slashdot.org,slashdot.org

The leading dot on the first domain entry says to search all hosts in the domain.

Problem: GSA does not provide very good status of where or what it is searching. It only has a dashboard light to say it is crawling. No details.

Problem: We found that the GSA would get caught in an endless loop if it encountered a user website controlled by a database. It would endlessly follow the next and previous links to find every database entry.

Our university library subscribes to a number of electronic databases, such as, EBSCO PsychINFO, etc. The GSA indexed every possible look-up.

Our eval licenses was limited to 1.5 million pages. Some of these databases contain hundreds of thousands of pages. Solution: Those setting up their own web server must employ proper robots.txt files or risk having their entire server blocked from indexing.

Don't use GET to modify application state!

[Augusto]

The problem is not google, is the way your app is designed!

Universal Resource Identifiers -- Axioms of Web Architecture : Identity, State and GET

In HTTP, GET must not have side effects.

In HTTP, anything which does not have side-effects should use GET

If somebody visited your site with a pre-fetching tool like the google web accelerator, you will also find the "delete" button being checked automatically like this. Change those deletes to use POST instead.