Slashdot Mirror

+1

Yes, google beat altavista by using "Page Rank" (i.e. analyzing the graph of the way pages were linked together, to estimate popularity). It was pretty remarkable, altavista was clearly more flexible (like other people here, I remember the "NEAR" keyword fondly), but you had to read through a page or two of links to find what you wanted, and google had a knack for putting it up top in the first few links (hence the "I'm Feeling Lucky" button, for when you were pretty sure you wanted just the top link).

(Why do we need to explain things like this? Is everyone on slashdot these days really this young, or do you guys have no memory at all?)

What was interesting about this is the moment google became really successful, it choked off the behavior that Page Rank relied on: why bother linking to relevent stuff when you knew everyone could just google it up if they felt like it? (And it doesn't help that wikipedia opted-out of this to discourage spam links). And so, ever since then, google has had to keep dancing to different ranking techniques (e.g. relying on user click data) to try to stay ahead of the SEO scum. So yeah, google no longer seems quite so wonderful (and myself, I'm more inclined to use blekko.com or duckduckgo.com

And now we're stuck in a world where you always get shown the popular stuff that you're expected to want to see, so the popular stays popular, and struggling upstarts have to struggle even harder...

If you want a "user experience" with someone second-guessing you and tossing extra keywords into every search, pfft, google it.

I occasionally try new search engines ( Google remained my favorite ) yet recently switched, due to proof that one is better... for me. I'm a scientist. I was convinced by the results of the game, Three Engine Monte, over at http://blekko.com/

" search term /monte "

I was impressed by how often I picked the Blekko search results link. Most often, the more relevant listing was unearthed by Blekko. I found better information with Blekko. I was mightily impressed, and switched. Unless you want local listings every search on a movie title, (which still seems intrusive to me), in which case stick with the big brother who gives you priority paid listings.

Grasshopper, if you are not trying new search engines, regularly, you are <strike>eating search results pablum</strike> missing out on some awesome information.

That makes no sense. Facebook like button really isn't on all pages of the internet because it makes no sense to have it on them, and isn't as widespread either. But every site wants to see how many visitors they have and all other information about them.

But hey, if you don't believe me, take a look at Blekko's Grep the Web. As part of crawling the web they do exactly this kind of stuff, to determine how many websites have something compared to other.

Here we can find number of domains with Google Analytics: 12,380,670
Here we can find number of domains with Facebook like button: 522,242 + 817,817 = 1,340,059

Yep, exactly the same. Except that Google Analytics is installed on 11 million more domains than Facebook like button.

That makes no sense. Facebook like button really isn't on all pages of the internet because it makes no sense to have it on them, and isn't as widespread either. But every site wants to see how many visitors they have and all other information about them.

But hey, if you don't believe me, take a look at Blekko's Grep the Web. As part of crawling the web they do exactly this kind of stuff, to determine how many websites have something compared to other.

Here we can find number of domains with Google Analytics: 12,380,670
Here we can find number of domains with Facebook like button: 522,242 + 817,817 = 1,340,059

Yep, exactly the same. Except that Google Analytics is installed on 11 million more domains than Facebook like button.

That makes no sense. Facebook like button really isn't on all pages of the internet because it makes no sense to have it on them, and isn't as widespread either. But every site wants to see how many visitors they have and all other information about them.

But hey, if you don't believe me, take a look at Blekko's Grep the Web. As part of crawling the web they do exactly this kind of stuff, to determine how many websites have something compared to other.

Here we can find number of domains with Google Analytics: 12,380,670
Here we can find number of domains with Facebook like button: 522,242 + 817,817 = 1,340,059

Yep, exactly the same. Except that Google Analytics is installed on 11 million more domains than Facebook like button.

https://blekko.com/ is an example of a competitor which has its own index.

Then I realised what I wished for was something like Yahoo.

I think what you want might be this: http://blekko.com/

I'm sure it's much more complicated than this, but off the cuff: You need a manual process to find cases like Penney's. Then, when you find a Penney's, you see all the sites linking to Penney's and they immediately become suspect. Not all of them will be selling links, but a lot will be. If you find a few Penney's's you start to build a spamrank(tm), narrowing in on sites that use stuff like TMX. You make outbound link weight inversely proportional to spamrank(tm, remember), and when you cross some line in your spamrank your outbound links become invisible altogether. Permanently. Additionally, the spamrank would add up like pagerank does on the target site and you make spamrank, say, 10x the weight of pagerank. You buy links, you get punished.

But it sounds like Google hates manual processes, they want to fix the algo. I don't see how that's possible without some crazy AI stuff going on (not that they couldn't go that route, mind you). Whereas I (a person) can look at a page and immediately say "link farm," doing that with a computer would likely be crazy difficult. Mostly because the best spam sites are legit sites, they just also sell links.

Speaking as a small business owner it's frustrating as hell. We've tried going to 'SEO' route, but A) there are a ton of super shady businesses out there selling this crap, and B) THIS IS NOT THE WAY IT SHOULD WORK. It's annoying when Blekko has us #1 for almost every related search term, but on google we don't even hit the first page for half of them. And if I take a handful of the people above us, scan their inbound links, the vast majority are all paid links. ARG. (Not that I think blekko has a better long term strategy, I think it's just as easily gamed, it just hasn't been... yet.)

I guess we just need to get as big as stackoverflow and complain, that way we can get customized changes. /END RANT

I've been hearing a lot about blekko lately.

Haskell (the language) and GHC (the compiler) were both already going before MS joined, yes. But since when do you have to start a project to innovate in it?

I did not say MS does not innovate at all, what I did say was "Almost nothing that is innovative comes from MS." I then went further and said MS either "bought, or like Steve Jobs, copied or stole then brought to the masses."

Other than new methods of limiting competition and spreading FUD, MS has not innovated much in-house.

More broadly, just go to Google Scholar and search for "Microsoft Research".

I did and got about 598,000 results. I next googled Google Research and got 3,120,000 results. To make it more balanced I'll try another search engine too.. Wow, Blekko returned 89M for Microsoft Research and 145M for Google Research. Ah, go ahead and try MS's Bing: Microsoft Research returns 73,100,000 whereas Google Research returns 70,800,000 results. On 2 on out of 3 search engines "Google Research", without the double quotes, returns more results than "Microsoft Research".

Now I'm not sure if searching for "X Research" means much. It's actually start-ups that do most of the innovating. Established incumbents then either try to buy them out or copies them. Just look at web browsers. After Internet Explorer (IE), licensed by Microsoft from Spyglass Inc (which is another story), won the browser war in the '90s MS stopped improving IE. Internet Explorer 6 was released on 27 August 2001. It wasn't until 18 October 2006, more than 5 years later, when Internet Explorer 7 was released. So it wasn't until open source Firefox started gaining marketshare before MS released a new browser itself.

Oh and about Spyglass Inc. MS licensed the source code to Mosaic and agreed to pay them a quarterly fee plus a royalty from Microsoft's Internet Explorer revenue. By including it free with Windows though MS thought "they did not have to pay royalties to Spyglass Inc". So Spyglass sued Microsoft before MS finally agreed in a deal to pay Spyglass $20 Million.

Falcon

Haskell (the language) and GHC (the compiler) were both already going before MS joined, yes. But since when do you have to start a project to innovate in it?

I did not say MS does not innovate at all, what I did say was "Almost nothing that is innovative comes from MS." I then went further and said MS either "bought, or like Steve Jobs, copied or stole then brought to the masses."

Other than new methods of limiting competition and spreading FUD, MS has not innovated much in-house.

More broadly, just go to Google Scholar and search for "Microsoft Research".

I did and got about 598,000 results. I next googled Google Research and got 3,120,000 results. To make it more balanced I'll try another search engine too.. Wow, Blekko returned 89M for Microsoft Research and 145M for Google Research. Ah, go ahead and try MS's Bing: Microsoft Research returns 73,100,000 whereas Google Research returns 70,800,000 results. On 2 on out of 3 search engines "Google Research", without the double quotes, returns more results than "Microsoft Research".

Now I'm not sure if searching for "X Research" means much. It's actually start-ups that do most of the innovating. Established incumbents then either try to buy them out or copies them. Just look at web browsers. After Internet Explorer (IE), licensed by Microsoft from Spyglass Inc (which is another story), won the browser war in the '90s MS stopped improving IE. Internet Explorer 6 was released on 27 August 2001. It wasn't until 18 October 2006, more than 5 years later, when Internet Explorer 7 was released. So it wasn't until open source Firefox started gaining marketshare before MS released a new browser itself.

Oh and about Spyglass Inc. MS licensed the source code to Mosaic and agreed to pay them a quarterly fee plus a royalty from Microsoft's Internet Explorer revenue. By including it free with Windows though MS thought "they did not have to pay royalties to Spyglass Inc". So Spyglass sued Microsoft before MS finally agreed in a deal to pay Spyglass $20 Million.

Falcon

Blekko won't work

There is one other thing in this world that search engines take care of other than advertising. Do that right, and you win a whole load of market share relatively easily.

The funny thing is, Blekko is doing a hell of a lot better than Google is in this regard. Links NSFW.

These Slashtag things aren't a new idea. Just a new name to the tagging system of old. Applying it to a search engine and mashing it with twitter @'s and facebook Likes isn't even that innovative, at least not something we didn't see coming. The fact is Google is too busy trying to take over the social aspect of the internet instead of simply integrating with the existing systems in place. They probably dropped the ball on this one and are starting to look a lot more like Microsoft these days.

People in power with all the information in the world are control freaks. Whuwuldathunkit. News at 11 I guess.

I do try the other search engines every now and then, but even when searching for something rather obscure Google returns more relevant results than the others. Seriously? I haven't noticed any limitation in the amount of ground that blekko.com or duckduck.go cover... and it really doesn't matter if there are a bajillion pages you index when two thirds of them are spam pages with content ripped from wikipedia.

Also, when I search for something really obscure, google always wants to push me in the most obvious direction, which is not typically what I want...

Making good searches simply is something one must learn, no search engine can read your mind and find exactly what you mean. Tell us more about this fabulous skill you've developed. We're all new to this internet thingie here.

One of many search engine features I can think of, but have never seen implemented, would be to treat different grammatical forms as rough synonyms. It's a little irritating to have to run a nearly identical search multiple times, once on a term like "indexes" and again on "indicies" and again on "indexing".

Yes, I switched to blekko.com as my default search engine in firefox recently: blekko.com. It works well enough that I don't touch google very much any more. Slashtags are a really interesting feature.

Certainly duckduckgo.com is decent, but it's js features are off-putting to me (though not as bad as google's new ones-- a drop down of guesses you don't want that obscures the button you want to click... I thought google tested things before roll-out).

The central problem that any search engine needs to deal with these days is the rapidly declining quality of content on the web. When google got started, it could use the web of links as a guide to quality, but google's success choked off the behavior it relied on originally: no one bothers to link farm any more. And say what you will about the democratic nature of the blog revolution, but there's a downside to making it possible for almost anyone to publish something on the web.

I keep hoping someone is going to put over a fad for making things actually work well... "Web 3.0: now it's serious".

>>>Majority != Monopoly.

I said "near" monopoly and 80% or higher certainly qualifies. In the U.S. 80% is high enough to amend the highest law of the land - the constitution. 80% was high enough to break-up Standard Oil and ATT.

Anyway I'm sold - http://blekko.com/ws/+/press-videos?h=1
I used to use metacrawler which was once a decent engine
"Windows Live" is different from Bing?

politics /retarded:

You do not have a slashtag called /retarded. Do you want to try:

politics /politicalblogs
politics /conservative

Impressive.

On the topic of reality distortion fields: jobs /apple versus apple /jobs

On the topic of reality distortion fields: jobs /apple versus apple /jobs

... for example. "Climate change /science" will restrict your results to hits from scientific Web sites.

Massive failure on that example unless you consider the top three results (newscientist.com, livescience.com and physorg.com) to be more than just news sites. And (of course you new this was coming) the gold standard does a better job with the same search.

Of the first page of Blekko results, I'd argue that only half of them have any business being on there. The other problem is that a lot of things like date ranges or news that this slashtag hopes to fill is already covered by Google's advanced notation. People who need these have probably already learned to use them (for instance the site:slashdot.org term helps me see if a story has already been up on a topic). If you want a bias other than range restrictions, just add it as a search term.

I spent a lot of time playing around with this and nothing I tried really jumped out at me as "useful." Of course I was just fiddling around and not really looking for anything in particular.