Slashdot Mirror
The Un-Google - The Search Competition
WinEveryGame writes "The Economist is running an article on the state of the competition for Internet Search. While Google clearly dominates, and continues to have positive momentum, its leadership is still vulnerable. The search-engine battle is not over yet." From the article: "In terms of momentum — mass times velocity — Google's lead indeed looks daunting. It has by far the most mass, with an American market share of 43% as of April, which reaches 50% counting AOL, an internet property that uses Google's search technology. This compares with 28% for Yahoo!; 13% for MSN, which belongs to Microsoft; and 6% for Ask, which is owned by IAC/Interactive Corp, a conglomerate of about 60 online media brands. Google also has velocity: its market share grew by 17% in the four quarters to this spring, whereas Yahoo! and MSN both lost share. Only Ask has more velocity — its share grew by 35% — but then again it has little mass."
There are some customers (government/military included) that are aware of the two concepts of precision and recall. Before you groan and skip this post because you recall those words from all classifying algorithms, you should take note that there are two stages we have yet to meet in this respect.
One is simply improving precision without sacrificing recall. When I search for 'horn' in Google, how many of those searches are relevant? I was thinking about a French horn (instrument) and the first link brings me to a society about them. The next three links, however, do not. You might say, "Well, gee, you should have put 'French' in your search" but is this really necessary? So there is some money to be made in "learning" search engines that tailor themselves to the user or perhaps the results could be displayed intuitively in domains of knowledge (a la Clusty). So that I can select a node that applies to the correct searching term and see all results returned below that. Have you ever wished to view your search results in a format other than a linear display of ranked results? The documents are related in more than one dimension, you know. As computing power increases, I suspect there will be room to display them in two dimensions (heat/area mapping, nodes & vertices on a plane) and three dimensions (spatial 3D engines with nodes & vertices in space).
The second stage is giving the user the power to adjust precision versus recall. Even a graphical interface that shows the F-measure relationship between precision and recall would be helpful to consider in the search engine wars. Say you give the user some control through a slider AJAX interface of a threshold ß. But the threshold isn't simply the "Google score cut off" or even a term frequency cutoff. Instead, it's applied to be a "relevance" threshold. You would score relevance by fingerprinting frequency, specificity, clustering and other useful tools by using a domain ontology or taxonomy.
Another big thing that is missing is identifying what kind of data you are searching. Social data? Scientific data? Historical data? etc. Perhaps I'm only interested in who's who to Stephen Hawking. I'd search for him and flip through nodes of separation from him to other people.
The current search sites also only tend to favor key-word regular expressions. What about searching with raw text or entire paragraphs? If you want to see an interesting demo of this, visit Collexis' Demo Site which alludes to a whole new kind of searching.
The key to entering the market as a competitor with Google is to pick up Google's slack and to try to pose yourself as a complimentary service to Google. Google is terrible at closed domain searches but amazingly efficient at open domain searches. You don't want to compete with them so fill a different part of the market. Google benefits from simple design, so go to an advanced flashy complex design. Most people aren't looking for that but the people that are have nowhere to go.
The Economist is alluding to potential leadership problems inside Google. Who cares? That's not going to be Google's downfall. Google's downfall will be an new intuitive way to search and the only thing that will prevent their downfall is if they buyout the company or bone up on the technology.
The search-engine battle hasn't even hit its stride.
My work here is dung.
It should really be m*v^2. (m*v is a vector equation, m*v^2 is a scalar)
It seems more and more when I try to find something on google all I get are a bunch of link farms. This morning I was trying to find a bike jersey for a friend of mine and on the first page of results, it took getting to the second page to find any actual results. I did much better using Yahoo and found what I was looking for on the first page of results.
This is just one example, but it happens constantly...
The reason I mainly uses Google is Google groups; whenever I have a computer related problem, the Usenet archive is often helpful. I do not think that there is any other archive of Usenet like this out there (available for free). If there are, please share the links.
And how many slashdotters find Google Groups useful?
Remember the year 2000? They promised us flying cars. They delivered the PT Cruiser...
Actually, I have as small personal page for people who are looking for me to find me. It is here. I am probably the only person in the world with a page that lists my name and my elementary school name. If you do a search on these terms "JOSEPH COTTON SEABREEZE" in google, you will not find my page. If you do a search in yahoo, then there it is at the top. So Google is not king, by any marker other than market share.
Well, actually, almost all current search engines suck. There is waaaaaay too much noise in the results they return. Let's say I'm doing a search for "product X", search for it in Google and what do you get? Several links to ebay (which may or may not be current), tons of links to various "rate it" sites such as epinions/nextag/msn/etc, and maybe a few smatterings of other sites mixed in. Typically the manufacturers own site won't even appear in the first couple of dozen results!
So basically, I agree with the general position of the article, that there is still a TON (actually several tons) of work to be done and room for someone else to move in with a truely superiour solution. While it's great that Google is tinkering with lots of other technologies, I wish they'd actually make some real advances in their core business (and actually, I'm slowly starting to come to grips with the fact that their core competency may not be searching, but really it's in creating low latency widely distributed computing infrastructures). For all the years and the massive sums of money, my search experience is not significantly better than it was 5 years ago.
I was on the phone with some engineers at MS the other day and even they admitted that they use Google. It's just better... for now.
http://religiousfreaks.com/Does anyone know how they calculate these market share values? AFAIK they don't all publish traffic statistics.
Developers: We can use your help.
Which world do these numbers come out of? This month on my private site so far I got 1400 incoming links vom Google and 30 from MSN (the next runner up), 27 from Yahoo. Maybe it's just that Google loves my site for some strange reason, but I can't imagine my own little sample of web hits is statistically so "off" from their numbers. Other sites I admin for have similar numbers.
The numbers of pagehits by spiders from those search companies are much more on an equal basis. Sometimes one of them is on top, sometimes the other, but they all spider like crazy.
Much more interesting are little search engines like gigabot, which never ever gave me one incoming link but still spider like it's going out of style. Somehow makes me think they must live either off warm air or spam. What reason to be do they have?
I don't know how they measured the dominance of these search engines, but I know how its supposed to be done: by the number of hits it gets on Google. In that respect, Google: 9,630,000,000 Yahoo: 5,240,000,000 MSN: 4,220,000,000 Ask: 2,140,000,000 Clearly, though, the most dominant search is the word "search" itself. It gets 16,670,000,000 hits.
"This compares with 28% for Yahoo!; 13% for MSN, which belongs to Microsoft; and 6% for Ask, which is owned by IAC/Interactive Corp, a conglomerate of about 60 online media brands"
This isn't over, simply due to lack of certainty in net neutrality. If media companies get leverage to control bandwidth to the big search companies (Google), it goes without saying that that these figures will change significantly. For Google, it could be death by a thousand cuts...
It's not a war, it's a market. Markets change. That's a good thing. Right now most people are happy with google. If something better comes along, I am sure nobody would stop embracing it just because they love Google so much. For shareholders and Google's owners it's probably different, as they want to keep on top and gain maximum market share to increase profits (or whatever). But as a consumer I couldn't care less if the best search engine is called google yahoo or msn. If a search engine searches well, I'll use it. If it doesn't - well, game over.
The article totally fails to take Lorentz-FitzGerald Contraction into account.
Intron: the portion of DNA which expresses nothing useful.
This is were Google truely dominates, in cult of personality.
It seems more and more when I try to find something on google all I get are a bunch of link farms. This morning I was trying to find a bike jersey for a friend of mine and on the first page of results, it took getting to the second page to find any actual results. I did much better using Yahoo and found what I was looking for on the first page of results.
I speak of the same experience in another thread. Simple fact is, Google appears to be doing nothing to make searching truely better. I give them some benefit of the doubt by saying "appears" since they may have "the next great thing" in the wings and we just haven't seen it yet. But lets face it, searching today is a time wasting experience. What's worse is that several years ago, it was time wasting due to the large number of seemingly random links. Well, those links aren't so random anymore, AAMOF, those links are geared to simply make Google money. I've said it before and I'll say it again, anyone who buys into their "do no evil" marketing fluff can give me a ring, because I have a nice bridge to sell them. How are they any different that M$ in this regard? They have tons of money, some of the brightest minds in the valley, and yet a simply search for a product gives me pages of utter crap. Hell, if a manufacturer makes the mistake of naming their company after their flagship product, you won't even be able to find their main website in a search until the third page!?! But Google still collects their bucks. How is this helping? How is this not evil?
Anyway, off my soap box. Here's to hoping someone can come along and actually do some good here (and yes, it may even be Google themselves).
Obwan: that search engine is our last hope
Yoda: no, there is another
(later)
Yoda: google... page rank is strong with you... pass on what you have indexed... there... is... another... search engi(ugh)
This way you can see how much energy a company has, and thus it's importance (m*v*v/2), growth can end *very* fast.
The energy a company gains is m*a*v, that shows that google is the best growing company...
etc.
Simple for Ask, experts are those that best further the marketing goals of their backers.
Stop intellectual property from infringing on me
About a month back I did a comparison of some searches, based on ask's claims of understanding concepts. They were able to outperform Google in this functionality: http://www.nirajsanghvi.com/stories.php?sid=318&ti d=55
and that's a good thing.
The barrier to entry in search is moderately high (you have to be able to afford the hardware to do your indexing) but there will always be people willing to invest in search. It's easily monetized (love that word, eh?), there's no cost for users to switch to new competitive product, and there's no magic bullet that gives you both accurate results and the ability to weed out aggregators and shady SEOs. As long as developers can come up with new search algorithms that give better results, there's the chance that the "next Google" could be launched.
I like and use Google, but that's because the results are usually valuable to me and the ads are minimally intrusive. Currently, the one issue I have is Google's inability to prevent aggregators from showing up in search results. I've never found anything useful through aggregator pages, and I'd like Google to filter them out.
Anyway, the ability of new companies to explore search is something that's good for SE users. New search startups can be launched and attempt to improve search. Google is forced to innovate. Where's the downside?
Interested in a Flash-based MAME front end? Visit mame.danzbb.com
At some point within the last few months, Google removed the "Remove this site from search results" button. I made heavy use of it, perma-banning resultspammer sites like ExpertSexchange.com and its ilk.
Sadly, this button is now gone from Personalized Search, and the resultspammer sites are steadily reducing Google's usefulness to me. Where I could once search for specific tech terms and get a good batch of reference resources, now I'm getting junk portal pages for the top five results.
Sure, I could report a link as spam, but that's a lot more time-consuming than the button, and it doesn't appear to have any immediate results for my searches.
This makes me sad. I've loved Google since I first met her, but I can't be with her if she's going to continue mainlining spam.
http://unxmaal.com
The reason everyone still uses Google isn't because they have the best ranking of results anymore, which is usually encrusted with spam sites designed to beat Google, but because they have the most COMPLETE results. When you search for something rare, Google most likely will return results no other basic engine has. So people have gotten accustomed to checking Google first out of habit more than anything else.
To me, I think the future of search isn't necessarily a better Google, but something different. The problem with Google is the same as its strength - its simplicity. There is very little control on Google for more complicated searches, such as searching only company websites, or searching only encyclopedia content. It's just a big kludge for them to add stuff like travel info or weather or movie info without knowing the intent of the searcher beforehand. Searchers have to get savvier, not just the algorithms. I think search aggregating sites like Seaurch.com which has 200 engines but still uses a simple interface, is a great idea. Sites like Clusty.com also take an interesting approach towards understanding the searcher's intentions.
There are no actually subject specific "experts" at Ask. They use computer algorithms which are generic enough to work with any subject group, not just ones that a single person might know a bit about. The term "Expert" is figurative and not literal.
The parent AC is correct.
.edu sites who are there principally to educate rather than sell you something. If you have to do general search, add
Big business turned the Internet (the WWW part principaly) into little more than 'online TV'. Because of all the $$$ at stake, we have probably the best search engine around, Google, drowning in ad driven/cash driven search engine spam ('spamdexing').
If you are searching for something in Google, add site:.edu to your search. By doing that. that should lead you to
-shipping -visa -mc -amex and the like to your search terms to block the 'ad pages'
Good Luck!
P.S. The best search engine would just index the single homepage page served up by the webserver at all 4,294,967,296 possible IPv4 addresses (minus the reserved/private/unused ones). The rationale is that if you are paying for webspace and a unique IP address, chances are good you have worthwhile content there. This would eliminate 'spamdexing' in all its forms in one bold stroke. The drawback is that 'online communities' would only be listed by the main IP webserver address and not by the URL which is why Google is in the 'mess' they are in.
If you are doing worthwhile research and want to avoid as much e-commerce as possible in the process, here again are some helpful tips....
add site:.edu in Google searches (mentioned earlier - restrict search to non-profit educational sites)
add -shipping -visa -mc -amex and the like to Google searches (mentioned earlier - helps eliminate product 'ad pages' and leave behind product review pages
search well-known 'info sites' like http://citeseer.ist.psu.edu/ or http://www.wikipedia.org/ If you can't find the info you want at those two places for free, the content you seek is either not on the Web or (more likely) you have to pay for it to get access. We all know how Net-savvy people hate to pay for anything they find on the Web....
As a last resort, search Google's Usenet interface at http://groups.google.com/ Helpful info can be found there but be prepared to really dig for it! =/