AllTheWeb Claims Bigger Index Than Google

An anonymous readers writes: "Hoping to attract more mass appeal for an online search engine with a cult following, Norwegian search engine AlltheWeb on Monday declared that it indexes more Internet information than longtime pacesetter Google. Boston.com has the story." Of course, pages indexed is not the only measure of a search engine and probably isn't even the most important.

Comments

ALL YOUR WEB PAGES ARE BELONG TO US!

[Neck_of_the_Woods]

Great, you have a huge index. I know a haystack that has more than one needle, but the stack is about the size of Texas.

Re:ALL YOUR WEB PAGES ARE BELONG TO US!

[Moonshadow]

Once again proving, it's not the size of your index, it's how you use it!

*ahem*

Simplicity not quantity.

[MarvinMouse]

Well, I think this might finally answer the question I have been wondering about my love of google for a long time.

Do I love google because it's so simple and easy to use with very quick download times and simple graphic interfaces, and good search algorithms that more often then not give me the sites that I am looking for in one page.

or Do I love google because it has a ton of useful sites logged in its database including all copies, half sites, under construction sites, etc.?

I am willing to say that's it's likely the first one, and I think that it might be that for most other people.

But either way, it'll be neat to see what AllTheWeb.com does well.

Re:Simplicity not quantity.

[SerpentMage]

Exactly. I did a search for "real estate Alsace" in the database and it found all of the ones that google did.

The only problem was it found many real estate sites within the US, and other places. And these places have absolutely nothing to with the Alsace

What I also love with google is that it will translate my english searches into the appropriate language search. So Real Estate becomes "immoblier", which brings up another ton of results.

It is the little touches that keeps me coming back to google.

Less webpages

[oever]

The ultimate test: how many webpages about me:

Google: 185
AllTheWeb: 57

I'll stick with google. It indexes more interesting stuff. ;^)

Re:Less webpages

[tenman]

I like the Image Search better than google's.

Obvious Ads--bleah

[dalassa]

Unfortunatly their ads are at the top of the page, followed by "top news" and then the links themselves.
However the first two returns for Scientology are the Scientology homepage and Operation Clambake. I wonder how long it will be before AllTheWeb is threatened.

Aside I'll need more proof that this thing is more accurate than google before I would consider switching.

Re:Obvious Ads--bleah

[Mwongozi]

Google is currently listing Operation Clambake first.

yeah but..

[SeicheWarning]

are they going to have any cutesy cartoons made out of their name for special occasions?

good not to have google monoculture

[kisrael]

Well, I was pretty happy with the results of a search on my name...happier than with Google in that once case, though that's but a single tiny datapoint.

In any case, it would be terrific to have a viable alternative to Google...despite Google's almost unnerving ability to do *so* many things Right, it is good to have somewhere to turn just in case something went wrong there. Not having a monoculture (which is what we're almost on the verge of with Google) is generally a good thing.

American Tech Companies

[Anonymous Coward]

The story says AlltheWeb.com is owned by a Norwegian company. Should people really support a socialist Scandinavian country? Any real American should only use capitalist homeland-based search engines, like Google or MSN Search. Like Bush says, "You're either with us or against us." Only a traitor would go against Bush's wisdom. What are you, AN AMERICAN TALIBAN?

Nothing to write home about...

[peterdaly]

I just did some searches, and it appears to be ok for finding information. Whether it's logic is as good as google's is hard to tell. Little slower than google. It doesn't look to me like there is any reason to use it over google. How many sites worth visiting are not in google's index?

This may be a case of a company picking a poor benchmark as their performance measurment. Google's draw is their great ranking logic, not index size.

-Pete

In other news ...

[Christianfreak]

Windows declares itself better than linux,
Gnome declares itself better than KDE
Emacs declares itself better than VI
PHP declares itself better than Perl ...

Let the flames fly :)

How does this work?

[Sludge]

I did some searches, and I ended up with different results than google. Perhaps of note, the results I got with alltheweb are from 1998, whereas google's are from this year.

Re:How does this work?

[Coolfish]

i did an even simpler test. I searched for "monkeys" - google returned interesting results all from different websites in the first 10 results. AllTheWeb returned 10 pages, SIX of which were all on the same domain, shacknews.com, and all of them had NOTHING to do with monkeys!

This search engine can't even find monkeys, then forget about it.

Enough with the Google worship, already.

[HEbGb]

God forbid someone presents an objective comparison between Alltheweb and Google. Responses such as "Google is my God" and Timothy's little snip in the article do nothing for anyone really interested in using a useful search engine.

I just used Alltheweb for some common searches I do, and you know what? It found a lot more useful hits than Google did. Yea, imagine that.

But Alltheweb didn't seem to have a cache, which I thought was very useful in Google.

So, come on, folks, give it a chance, and don't jump to conclusions without an objective analysis. The tendency to blindly worship things like google/linux/linus/transmeta is far too common on this site.

Re:Enough with the Google worship, already.

[Gutboy_Barrelhouse]

Timothy's little snip offers at least as much as your post. He's just pointing out that pages indexed is (only) one measure of a search tool. That's a starting point - what are some others? Another poster mentioned up-to-dateness of the pages returned. That's good. Clean interface is another one. Features like a cache, which you mention, are another. Response time, configurability, documentation, what else?

Other than that you just say that alltheweb gave more hits for you on a couple of searches. That's a pretty useless measure of search tool quality to anyone else.

For my part I'm not sure what would make me switch from Google. I really value their interface, and none of the recent challengers are preferable to me in that regard. I switched TO Google because IMO it was a big leap in result quality over the earlier generation, but until we start using personalized intelligent agents, I'm hard pressed to imagine how a new engine could produce as big a leap. Basically, as search engine quality improves, for most people the sample of searches that they'd have to do in order for a comparison to produce clearly discriminable results also grows. Will very many Google users really sit down and do like 50 searches with both it and AllTheWeb, on the chance that ATW proves slightly better? I doubt it, and I'd hate to have to be in the planning room of a Google competitor, trying to think of the killer app that could get people to switch with just a few searches.

Though I'd certainly switch if I could have the Librarian from Snow Crash.

Targetted searching is needed, never mind size

[eXtro]

When I saw Gladiator I was wondering how much of the story was actually historically accurate. I don't own an encyclopedia so I did a bit of research on the web. I looked up some of the main historical characters, Emperor Commodus, Marcus Aurelius and others, using google, altavista and others. What I found was that the spike of interest in the movie had completely swamped out any historical results. I had to wade through pages and pages of hits before I finally started getting to information that was useful in my context.

Google is my favourite search engine, even now, its ads are unobtrusive and don't pollute the search results. They've been good net citizens and they've done substantial research into how to better search. There results are typically the best as well.

In this case their search results were very broken however, at least for the purposes of my search. What I'd like to see is google, or an engine as effective as google, add in the ability to constrain your search to subject areas. In this instance I'd constrain my search to historical sites and would have received mostly uncorrupted hits. This is different than a web directory. Web directories don't classify sites based on there quality. Google does in a round about fashion, it lists sites with more people linking to it higher than sites with less links.

I'm not sure how the details of this would work, self-nomination would not necessarily work. Porn companies would gladly pollute the keywords on the off chance that somebody looking for history would buy a membership to their site. Letting individuals vote a site into or out of a keyword might work, though you'd be in danger of concerted efforts to say vote out anti-Scientologist information and vote in pro-Scientologist information when both actually could be under a religious keyword.

Anyway, linking to more sites isn't necessarily helpful in my opinion. What I'd prefer is the ability to narrow the focus of my searches.

The true test

[Mr_Silver]

Search for "sex":

Google: 63,500,000
AllTheWeb: 25,435,205

I think I'll stick with Google :o)

Cached here

[GothChip]

For those who are unable to reach AllTheWeb here is the homepage through the usual Google cache.

Major, Major Flaw

[limekiller4]

I just tried to pull up one of my own pages with this engine. Got:

"Redirection limit for this URL exceeded. Unable to load the requested page."

Which, as near as I can tell, is their way of throttling commercial hits. Wonderful. Moving the mouse over the link doesn't reveal the address in the bottom bar, either, so the only way I can think of to obtain the address of the item it matches is by right-clicking and selecting 'copy link address', opening a new window and pasting it it (and having a browser that is capable of doing this), then editing the URL so only the target link text remains.

You can't even right-lick and open in a new window to do this. If you try, you get "about:blank" which, afaik, means they're using javascript.

These people sure go through a lot pains to render a result and then not let you anywhere near it. Saying they're bigger than Google is a bit like someone bragging about how their PDP-11 is bigger than my Athlon. Cripes.

AllTheWeb _has_ one advantage

[rainer_d]

Even if you totally discard it as a Web-Search-Engine, the FTP-Search is still one of the best.
It was hidden as ftpsearch.lycos.com for some time, but now it seems to have come "home".
BTW: the last time their OS was visible through the firewall, it was FreeBSD...

Anyone remember archie ?

How about the image search then?

[Jugalator]

I'll just say this:

Google manage to get a graph of the slashdot effect among the first 20 hits, while AllTheWeb just manage to get Cliff showing a Think Unix book (in weirdo hawaiian clothes). :-)

I don't know about you, but Google give me more relevant matches as usual. ;-)

Re:Where's TEOMA?

[Jugalator]

lol

Slashdot.org not among 10 first matches when searching for "slashdot"...

Needless to say, I never wasted more time by checking the next page...

The old saying holds true:

[nemesisj]

It's not how big it is, it's how you use it.

Google is still way more useful in my opinion.

Vote Google

[salmo]

The reason I'm for Google has little to do with technology. It has everything to do with advertisements and capitalism.

I'd rather support a company that uses subtle advertisements like Google does than a company that uses in your face banner ads, etc. (Then again I'm posting on Slashdot!) Also I make a point to check out the ads evey now and then on Google and visit the company's site. I may be getting hosting from an advertiser on Google soon.

If people who advertise on Google make more money than they do with banner ads, pop-ups, etc. then we'll see the idea spread. I don't like in-my-face ads, so I do what I can to tell companies that. It's called being a responsible consumer.

Plus more valid hits come up when I search for myself on Google ;-).

Your web server is broken

[jesser]

Redirection limit for this URL exceeded. Unable to load the requested page.

That is a Mozilla error message (source) and does not come from alltheweb. Your web server is broken. http://www.kaosinc.com/jen.shtml redirects to http://www.kaosinc.com/index.shtml, which then redirects to itself. This happens regardless of where I find the link to http://www.kaosinc.com/jen.shtml, or what browser I use to load it. IE appears to just sit there, Opera bounces between various stages of trying to connect, and Netscape 4 gives up after a few redirects and displays a raw 302-found page ("The document has moved _here_") without redirecting.

Moving the mouse over the link doesn't reveal the address in the bottom bar, either, so the only way I can think of to obtain the address of the item it matches is by right-clicking and selecting 'copy link address', opening a new window and pasting it it (and having a browser that is capable of doing this), then editing the URL so only the target link text remains.

An easier way to see the URL of the link is to hold the mouse down over the link, and then move off of the link before you lift the mouse button. But I still get the infinite-redirect error message if I type your URL directly.

You can't even right-lick and open in a new window to do this. If you try, you get "about:blank" which, afaik, means they're using javascript.

If I right-click on a link from the alltheweb search results and select "open link in new window", I see http://www.alltheweb.com/go/1/H/web/http/www.kaosi nc.com/jen.shtml in the location bar and get the same error message. What version of Mozilla are you using?