Learning to Love the Panopticon

mitd writes: "Cory Doctrow has written an insightful article about Google, search engines and how he stopped worrying."

Comments

Eschelon?

[ZaneMcAuley]

He spelt Eschelon wrong.

Anyway, I use copernic 2001 pro. Never touched a web engine for ages :D Best ever search client.

Re:Eschelon?

[JediTrainer]

He spelt Eschelon wrong.

Does anyone else find it amusing that we're obsessing over the name and spelling of a system that we don't even know (for a fact, with proof) exists?

If it does exist, how do we know its name?

If it doesn't exist... well... who cares what it's called?

Re:Eschelon?

[ZaneMcAuley]

It was based on the most common spelling used within the EU reports and from other web reports.

Re:Eschelon?

[Sandor at the Zoo]

He spelt Eschelon wrong.

That's OK -- /. spelt his name wrong. :-)

It's Cory Doctorow, not Doctrow.

The Switch

[Covant]

I remember a few years ago, when I made the switch to Google. I was impressed from the get-go, and have never looked back. Everyone I talked to, everybody who was using some other search engine, I turned them on to google. (It wasn't hard)

And now, in some places, rather than saying "do a search for [something]" people say "google-search it" (even if they don't use google).

You know something's great when people make a verb out of its name.

Re:The Switch

[cyborch]

I don't know why, possibly it's the lack of web portal-ness of google, but very few non-geeks I know use google. They alle stick to the local Yahoo clone.

I may be missing something, but I really can't see the reason why... could anyone enlighten me, is there something geekish about google? Or is it just me thinking that non-geeks want to use more bloated and less efficient solutions than geeks?

Re:The Switch

[Covant]

I'm pretty sure it's similar to the "use windows because it's on my machine when I buy it".

"I'll use this search engine because that's what appears when I install AOL, @home, sypmatico, etc."

That, and most people can't be bothered remembering more than two web addresses. www.hotmail.com (being replaced by simply using MSN messanger) and www.my-favourite-porn-site.com.

Re:The Switch

[Ed Avis]

Perhaps Google doesn't do such a good job when searching on non-geek topics. I use the Web mostly for computer stuff and random urban legend / Kevin Bacon searches, so I wouldn't know. But maybe if you want to book a holiday a semi-automated index like Yahoo does a better job than just counting links (after all, who links to a competitor's site?).

Re:The Switch

[sydneyfong]

Actually quite a lot of my friends use google or at least know that it exists. I think it won't be long before google really takes off into the public ;-)

Re:The Switch

[markov_chain]

I've been using the expression "just google for X" when referring friends to some site about X that I know shows up high on the hit list. It's funny, google is starting to replace DNS for me-- instead of remembering or bookmarking URLs, I just remember the keyword to google for. For example, the URL http://www.uk.research.att.com/vnc/ is harder to remeber than to just type "vnc" and hit "I'm feeling lucky."

I often wonder how much less productive I would be if google went away tomorrow :) If anyone from google.com is reading this, thank you!

Re:The Switch

[neuroticia]

Google is like the command line. It's fast, it's efficient, and it doesn't waste resources or cram trademarks down your throat. That's what geeks look for. Non-geeks very seldom look past what they already have and thus might never find google.

Re:The Switch

[Anonymous Coward]

Not to spoil your rhetoric, but Google has gone out of their way to index "geek" topics (Linux, Perl, MSDN, etc) and their plan was to use "geeks" as viral marketing agents. Probably significantly more important than the nice simple design in terms of their popularity with technical people. Content matters.

Re:The Switch

[neuroticia]

Perhaps you're right and it can be attributed to the higher incidence of geeky topics in Google as opposed to other search engines... But do a search on non geek topics.

Lipstick: 251,000 results.
Romance Novel: 471,000 results
Horoscope: 996,000

Sure, not as many results as 'PHP' and 'MySQL', but surely not paltry.

The same searches performed on AltaVista:

Lipstick: 288,076
Romance Novel: 141,561
Horoscope:1,073,632


Slightly more in all categories, however try the searches with PHP and MySQL as search terms in google and altavista (php:5,440,000/8,925,806 and mysql:1,980,000/16,577,970 respectively.) If anything Google has a lower tech:nontech ratio for search results returned. At least on the topics I've searched for.

The *QUALITY* of the results returned were higher for all categories in Google.

-Sara

Re:The Switch

[Anonymous Coward]

Guess what.. Who matter about the number of search results returned ? The ONLY important think IS the quality. And soon, people won't only replace "searching" by "googling" but also "quality" by "google like" :-)

I tell all friends to try Google, and all turned to Google. Did you ever tried the "image search" feature with your name :-)

Re:The Switch

[osu-neko]

Hehe! Yes, very true. I used to obsessively bookmark anything I might possibly be interested in revisiting in the future. Now I rarely bookmark anything -- only things I know I will be visiting at least once a week. And even then, it's usually after a few weeks of doing it that I finally bookmark it. Now bookmarks are just handy click savers, rather than a memory of useful sites. I don't need a memory of useful sites, I can always find them again on Google...

Nice description

[torqer]

In case you were like me and really had no idea what the submitter was talking about in his description...

The link is to an article that gives some insight into how google searches through the hordes and hordes of webpages. And bashes other search engines.

Note to submitter: while brevity may be the soul of wit try to remember we haven't read the article yet and need just a little more information.

Re:Nice description

[Accumulator]

Actually Google isn't the only search-engine using these techniques (that is: rank sites after how many pages that link to them). I have been on some lectures with FAST which Lycos and others are running their searches on. Perhaps they were first though, I don't know.

The main benefit Google has these days, is that they have ~8000 PCs clustered which they run the searches on, while FAST (as an example) has only 600. Google can therefore take the freedom to do searches that cost more processingpower, while others have to think of smart techniques to maintain good results without using the same power.

One example is that of searching for patterns, ie. several words in given order ("to be or not to be"). While Google uses their searchpower to find all those words, FAST saves all three words following each other ("to be or", "be or not", ...). This means three times the diskspace, but disk is cheap. This way, they have fast lookups, and save plenty of time.

There's a problem with this

[Tim Ward]

Now that Google will find anything you want so easily, isn't there a danger that people will stop putting links to useful and interesting sites on their pages?

I don't need to tell people, via a link, about some wonderful site I've found if they can find it for themselves quicker and easier using Google. So I might not bother to maintain my collections of useful links, and Google will lose its information source. A victim of its own success.

What happens then?

Re:There's a problem with this

[Cheesy Fool]

Why would it be quicker to load up google, do a search then click on the link (assuming its the correct one) as opposed to already being on your website and just clicking the link you've provided or just being provided with a link to a website?

Re:There's a problem with this

[rasactive]

Policemen don't have a job if there are no criminals. The job of a policeman is to arrest criminals to work exacly toward that goal. Police departments still seem pretty busy to me.

Re:There's a problem with this

[GigsVT]

I've thought of this myself. I know I don't do nearly as much "surfing" between related sites now that Google is here and works. I usually hit Google up, then if that site isn't what I want, I don't bother clicking their links section, I just go straight back to Google.

The one thing that may save us though is AOLers. Bear with me here. :) I think that maybe we have found the most efficient way to get the information we want, mostly because the novelty of the Internet has mostly worn off for us. We no longer spend hours bouncing from site to site, just reading random stuff. We use the Internet as a tool to expand our effective knowledge and intelligence.

This is obvious with the various Googlebots that have sprung up in lots of IRC chat rooms. This happens a lot in help rooms, if no one knows the answer, or doesn't want to take the time to explain it fully, they just !google and the bot returns the first link in the search.

So while people like us, if we were the only people on the net, would cause Google to fail, so long as there are still "surfers" out there, it should allow Google to remain meaningful.

Just my two cents.

Re:There's a problem with this

[ForceOfWill]

Then, it becomes harder to find things using google, and people start giving each other links again, and google gets better again.... see the cycle? I expect that there would be some sort of damping effect on this oscillation, so it would all even out in the end, with google being just short of good enough to warrant using it instead of passing links manually.

Re:There's a problem with this

[Anonymous Coward]

what a nerd you are. :)

Re:There's a problem with this

[guiding_knight]

In addition, what about all these /. links to google searches? Does google have a check in its programming to find links to itself? If not, as more and more people link to google searches, google could convince itself that it is the most authoritative site for any and every subject. I dunno about you, but I would find this very entertaining...

Re:There's a problem with this

[MrNixon]

Because with Google, you're pretty much guaranteed good results.... and youre not with Joe Webmaster's links.

Re:There's a problem with this

[Gleef]

As long as there are blogs, I think Google will have plenty of links to work from.

They may also start factoring in "The number of times people used a link on google" into the equation to make up for fewer links to work from.

Re:There's a problem with this

[bdktty]

Google has also started something new to help with this. They now have a toolbar that you can download, allowing you to google with out actually going to their site. The hook is, when you download the tool bar, you have the option of having every page request you make being sent to Google and archived for page ranking. They say there is no personally identifiable info, the link just gets copied to google....

Re:There's a problem with this

[abiogenesis]

One other problem would be choosing your favorite links from Google. If everybody goes that way, the top results for a keyword would continuously gain more links pointing towards them, and they strengthen their place in Google.

How to abuse Google

[AftanGustur]

Actually Google's system can, and is, beeing abused..

Re:How to abuse Google

[PeterClark]

Well, this has been known for a long time. But really, it's not as big a deal as one might think. "Scientology" as a search term pulls up an entire page of Scinetologist sites, except for #4, which is xenu.net. However, the first page for "Scientology secrets" is full of sites that debunk Scientology. So yes, the Church of Scientology has a virtual monopoly on the search "Scientology" but is far, far from controlling other search items. It all works out in the end.

:Peter

Re:How to abuse Google

[Chexum]

I'd not call it "abuse". It's simply that more pages (by real and virtual people) link to "real" scientology pages. After all, the COS is the source of information about scientology, don't you think? Telling this is the only job of google.

The same way, when you search for microsoft, you don't expect linux.org to come out at the top, and vice versa. In the COS case, the picture has more shades, obviously, but any serious research should be done not only on the first link. You can help the opponents by linking Scientology to xenu.net this way on all the pages you maintain, after all.

Re: How to abuse Google

[thebabelfish]

Slightly OT, but the Register has an article about using Google as an "attack engine".

Re:How to abuse Google

[Anonymous Coward]

This doesn't 'work out in the end'. The most likely query returns too many CoS results. That is a FLAW.

Google fails to narrow a search on 'scientology' to 1 or 2 of CoS official links, like it would with most other result sets (we're all familiar with the indented home/specific page pair format that google usually narrows things to).

So, this packed resultset is not an 'all works out in the end' result. Further, it happens to the most common query likely to be issued. Someone genuinely wanting to investigate Scientology before joining isn't going to do queries like 'scientology secrets'.

The "Crunchy Frog" sketch off Monty Python comes to mind on this. That's why HOW we report stuff can't be finessed. For a few people, the ' delicate taste of lark's vomit ' may not deserve a prominent warning label, but for most of us, we'd rather everyone get an eyes-wide-open view. What a soul then chooses to do with this knowledge is up to them. Eat, don't eat, whatever. But don't go through life allowing yourself to be easily tricked by the fine print.

Likewise, the top search results on ANY brand or trademark or corporation or religious entity should show a diversity and depth of opinions.

This isn't about Scientology for me. Personally, I couldn't care either way about that argument. I just don't want subtle predigested values implied in any research I do . I want my life difficult and complicated. I want to see glimpses of protesters and '.sucks' sites when I search for Pepsi or Congress or the Church of the Blessed Traveller or Stairmaster. And I'd probably conceed that 4th on the list is an acceptable position for the first anti-something site. Anywhere after page 1 isn't, though.

One last thought: Operation Clambake seems like a weak choice of 'warning' names, and the header lines listed aren't much more descriptive in the results I got when hitting google with a 'scientology' search. I would have glanced right past this site and not realized it was anti-anything. For a 'Danger danger, Will Robinson' sort of site, it's far too lacking in any loaded negative words like dangerous, manipulative, threat, etc. That 'Fight against Scientology' description line is the only vivid clue in the bunch.

.... posted anonymously because I really don't care about the CoS. But rumor has it They take these things waay too seriously and life's too short for me to find out. I also won't be posting traceably about La Cosa Nostra, the Russian Mafia, or that wierd guy in the mailroom. They might not care, but why take any chances...

Re:How to abuse Google

[Anonymous Coward]

When one entity knows how to spam a search engine, you can deal with it trivially.

When a million people know how, you have a problem.

Re:How to abuse Google

[Sloppy]

Whois information for the vast majority of these indicate identical registrations such as this one for exactscientology.net.

This suggests a rather obvious patch for Google's algorithm, no?

Won't .net screw Google all up?

[deadtreerus]

Won't all these "good" things about the net just get screwed up with the .Net age almost upon us? Seems like embrace and extend will cause a LOT of PROBLEMS with a simple yet elegant solution to searching for what you really want.

Re:Won't .net screw Google all up?

[labratuk]

Personally I think I will cope fine without .net. I won't use it, I won't develop for it, I won't encourage it. There's simply no need for it IMHO. Sounds like a lot of nice interesting little ideas (well ok maybe not so little) which are 'useful' in theory but they won't change the world. Remember Microsoft Bob? Weren't we all supposed to be surfing the net in virtual reality by now?

Hopefully time will prove me right on this one, but I doubt google will take .net to heart.

Come on, the net has coped fine with the current system for the last 15 - 30 years, I really don't think .net is that important.

Where's the magic?

[guerby]

In the age of DMCA, SSSCA, and angelic companies running after all those evil pirates in order to protect their beloved authors that deserve their protection, how comes no one has yet sued the biggest copyright infringer of all times ... the Google cache?

So where's the magic?

--
Laurent Guerby <guerby@acm.org>

Re:Where's the magic?

[Anonymous Coward]

Google deletes things from its cache (and actually stops indexing the page altogether) if someone threatens them. They don't monitor their content, so no-one can sue them up-front, and they remove without question anything that someone puts the pressure of a lawsuit on them to remove.

When you see a something illegal in Google's cache, it's just because the copyright holder hasn't contacted Google over it.

Re:Where's the magic?

[Jerf]

The short version: The DMCA makes provisions for certain caches used in the transmission of information, such as your ISP may use. There are certain defined procedures that the ISP must implement to allow people to get their content out of that cache.

Google implements those procedures, and claims protection under the DMCA for their cache. (Note the hoops you must jump through to get them to remove stuff are the legally mandated hoops under the DMCA; they are not trying to be nasty.) Now, a careful reading of the DMCA will show that Google probably doesn't meet the qualifications of this cache exception; but nobody has cared enough to fight it yet. The few who care just jump through the hoops and forget about it.

The long version is: Read the DMCA and compare against Google's DMCA page and decide for yourself.

Names and verbs

[SomethingOrOther]

You know something's great when people make a verb out of its name.

Or very bad (tm)
"Slashdotted and going "postal" spring to mind :-)

Re:Names and verbs

[CmdrTaco (editor)]

Or about pulling an "Al Sharpton"?

No human decisions ?

[EpsCylonB]

Do you think that the google search could be improved by more human decisions ?.

An example might be that goat.ce page (or whatever the url is) might get linked to a lot as example of bad taste (I seen a few pages that link to it and describe the page urging people not to visit it), which fine except that this web site is now getting linked to (or voted for which is how the google algorithm treats a link) yet it isn't a particularly good or informative website.

Even if someone was searching for something on bad taste, that page is not really an authoritive page about bad taste just an example of it.

Re:No human decisions ?

[Tablizer]

(* Even if someone was searching for something on bad taste, that page [goat.ce] is not really an authoritive page about bad taste just an example of it. *)

Sometimes a picture is, unfortunately, worth a million authoritative words.

Re:No human decisions ?

[ragmana]

But what human? A human decision based model would only make abuse easier and cheaper. Look at the criticisms of the Open Directory Project linked to at this post. The Church of Scientology easily abuses that human based system, while abuse of Google is more difficult and especially costly. Check the other posts on that thread too.

Not to mention the added cost of hiring Google editors.

Re:No human decisions ?

[NFW]

The moral of the story here is that whenever you come across a truly repugnant site, the last thing you should to is link to it. I mean come on... "hey, this site really sucks" will only increase traffic to that site.

So if you really think a site sucks, don't link to it.

advanced search

[Anonymous Coward]

"when searching was a spew of boolean mumbo-jumbo"

I still use AND, OR, and NOT ("-" in google)

Re:advanced search

[martyn s]

an AND is implied before every word that doesn't have any other boolean operator.

More Google Links

[Schwarzchild]

How Google Works

Undocumented Google Commands

Google Time Bombs

Google Science-Fiction

Re:More Google Links

[damiam]

And, of course, Googlewhacking.

Re:More Google Links

[hey]

How about using it in Klingon

Not all that Great

[gokulpod]

I do find Google good, but dont like people telling me all the time that is the ultimate search engine. People used to say that about Yahoo/Altavista before Google came along. And ow look where they are.
One thing that really jars me is that when I search for my name on Google, I find more links to amazon given to my own home page.

Re:Not all that Great

[SamBeckett]

It is that great, your homepage comes up #2. Presumably, Amazon.com is linked to more often than your homepage is.

http://www.google.com/search?hl=en&q=Gokul+Poduval &btnG=Google+Search

Re:Not all that Great

[Anonymous Coward]

Maybe you should change your name to not be
"Amazon Gokulpod"...

Re:Not all that Great

[Anonymous Coward]

Well, your amazon review actually contributes something to the WWW. Your homepage does not.

Re:Not all that Great

[Anonymous Coward]

he certainly contributes a lot of whining...

Sometimes I AVOID google!

[SomethingOrOther]

Google is brillient if you know what you are looking for. It finds the best pages straight away.
However, when I'm idely surfing (tm) I use something else.... I want to wander around the 'net not be taken straight to my destination.

Bit like driving somewhare along the back roads. You never know what you might find

Re:Sometimes I AVOID google!

[Anonymous Coward]

Maybe google could impliment a feature that allows you to search everything, except the words/topics you eneter!!

Rodney

sweet sweet google *drools*

[Anonymous Coward]

I too found it very weird that this search engine actualy found what i was looking for...

I have not looked back once i started.

Thank you google, thank you!

Rodney

encoding in DNA

[Alien54]

In a world of degradable storage, replicating copies is the surest way to guarantee longevity. Whether your data is in atoms or bits, the more copies you make of it and the more widely you disperse it, the greater the likelihood that your data will persist forever. (That's why Jaron Lanier jokingly proposed encoding printed matter into the DNA of the notoriously prolific cockroach, as a means of ensuring archives through a nuclear war and beyond.)

I can see some future biologist doing the the heavy work on decoding this now. And the arguments. of course, if it contained something like the Linux kernel, figuring it out could take awhile.

Heck I am still waiting for folks to find a licensing and copyright statement in the human genome.

;-)

Re:encoding in DNA

[Anonymous Coward]

Don't get your hopes up. The technology for encoding DNA is much too expensive. The cost per nucleotide base is about 50 cents, so encoding long sequences will get expensive quite fast. On the other hand the cost of replication is nothing once sequence inserted into some poor insect.

The other problem is that information quality will go down with time. Due to mutation and recombination effects. There is no evolutionary preasure on insect to keep mutation rates low regions of DNA that don't do anything. This means that all data will be lost within some time.

Re:encoding in DNA

[Anonymous Coward]

That's why they should use PARfile roaches!

I would go on worrying if i were you

[limbop]

Google works on the recursive principle that an important document is one linked to by a lot of important documents. search for "child pornography" and (i'm generalizing here) you're likely to find two kinds of sites: sites offering child pornography and sites opposing it. those will probably create two seperate cliques (if you look at the web as a graph) or clusters. It will be quite easy to offer them as two seperate lists both satisfying the search query. i believe northern light (http://www.northernlight.com/) does exactly this.

Now how about a similar principle for people? A suspicious person is one who communicates with suspicious people. If you have access to Email messages sent on the internet this is quite easy to achieve. Filter the messages to those mentioning "child pornography" and now do the same analysis as google does. voila! you are left with lists of child pornographers and of internet vigilantes. easy. automatic. you can start worrying again.

btw, if you are looking for an interesting technical description of the best search engine around, the original google article (http://citeseer.nj.nec.com/brin98anatomy.html) by Brin and Page does the job a lot better than Doctrow's.

Re:I would go on worrying if i were you

[GigsVT]

It is a little too simplistic, but it is totally feasible.

What about when a vigilante emails a bunch of sites flaming them and telling to take their stuff down?

This happens a lot in the spam/antispam world, antispammers probably trade more email with spammers than other antispammers.

Re:I would go on worrying if i were you

[limbop]

I was using an icredibly simplistic example on purpose. of course this process cannot be totally automated (if it were we'd have inteligent agents retrieving information for us) but it can be brought to a point where the amount of information can be handled by a human without wasting to much time and with high precision.

I like Google

[Anonymous Coward]

What I like most about Google is its simple interface and lack of tons of flahsing gifs and ads. I hate ads. One nice touch about Google is that they highlight various themes and holidays with very well done logos modified with cartoon characters for the occasion--Thanksgiving, Olymics, and so on. Whoever the Google artist is, he or she can be very proud of a job well done. Those cartoons are adorable, and I wish they had a gallery of past cartoons available for the browsing.

Re:I like Google

[La Buge]

Those cartoons are adorable, and I wish they had a gallery of past cartoons available for the browsing

Well, they have.

Re:I like Google

[Anonymous Coward]

me too :)

nothing new...

[illaqueate]

Vannevar Bush, As We May Think (July 1945)

Ben Schneiderman, Codex, Memex, Genex (December 1997)

Henry Jenkins, Information Cosmos (April 2001)

A puff piece with poor logic

[XDG]

The article boils down more or less to the following:

1. "Old" search technologies (Altavista, Yahoo) failed because they used approaches that found words but not content (Altavista) or relied on non-scalable human editorial judgement (Yahoo).

2. Google works (and is cool) because it uses available information about the number of links to determine (a) valuable content and (b) smart judges of other valuable content

3. The government efforts at creating the Panopticon will fail because they'll be stuck using "old" keyword approaches that can't pick out real content.

This argument is flawed in two key ways:

1. The author confuses the nature of the "search". Web searching is about finding *content* and the challenge is differentiating "good" content from "bad" content. Governmental "security" searching is more akin to traffic analysis and the goal is identifying dangerous *individuals* based on the content and pattern of their traffic. The challenge there is differentiating "good" (safe) speakers from "bad" (dangerous) speakers.

2. The author assumes (based apparently simply on opinion and what is popularly reported in the press) that the government will blindly apply "alta-vista style" techniques. His lack of fear of the Panopticon is based on an assumption of incompetence in the application of surveillance methods. Given the motivation and resources (both of which the government now has in spades), there is no reason to believe that more sophisticated and effective techniques will not be developed and pursued. Assuming Echelon has really been in operation, it's hard to imagine that, in the closed halls of the NSA, researchers aren't well aware of the limitations of keyword search and are far along applying cryptanalytical techniques to the real problem identified above.

It would seem that the author is trying to take advantage of hype and concern about government surveillance not to make a serious comment about it or whether one should truly be concerned, but rather to get an audience for his opinion that Google is really cool, which most of already knew anyway.

-XDG

Re:A puff piece with poor logic

[sam_handelman]

the challenge is differentiating "good" content from "bad" content. ... The challenge there is differentiating "good" (safe) speakers from "bad" (dangerous) speakers.

I agree with all else you say - including that the government has the resources to come up with new approaches to the problem - but I don't think that this challenge is really different from distinguishing between good and bad content. In so far as the government is trying to do what it shouldn't even remotely be doing, using this technology to identify subvsersives, you are right. However, in so far as carnivore might *actually* be used to intercept a criminal communique, I think that the challenge is very similar to what is faced by google.

Suppose that Inoccuous260@hotmail.com only ever sends one message, from some terminal in a public library, and it is the delivery schedule for a nuclear weapon. The best, most morally (if not legally) defensible use of Carnivore would be to intercept this message and hand it over to the Feds. If the Feds can do this, even once, Carnivore will be with us forever, however else it may be abused, b/c you will never rally the public will to end use of such a tool. The problem of identifying that message, and I don't want to brainstorm ideas here, but I'm sure we could come up with several, is very similar to the problem of picking out a biographical sketch of Allen Turing among all the sci-fi and hoopla, which Google can do using characterisation by links, and which the government would be hard-pressed to do without that human resource.

So, the author raises a fair point about the limitations on the "legitimate", let us say intended, use of carnivore. However, the unintended/illegitimate use, simple identification of dissidents, could indeed be carried out by a clever 10 year old, and is plenty worrisome even if Carnivore never does what it was supposedly intended to do.

Re:A puff piece with poor logic

[Anonymous Coward]

Absolutely agree. The article doesn't provide insight into either Google or privacy issues at all. Cory is a no-talent hack who can write good sentences, but that's about all. Look at the company he founded, OpenCola, he was supposed to be the CEO but they had absolutely no business model, and much more clearly so than other dot coms.

Suggestion to Cory: Stick to writing science fiction.

Wrong about email

[Karellen]

He's wrong about one thing. Email does have links. It has links indicating who it came from and who it went to. Even without the content, that sort of information, about who is talking to whom, and in what patterns, can be really informative to those who know what they're looking for.

If you include the content, it's a goldmine.

URLs embedded in email would make it better again

Aside from that though, great article.

Re:Wrong about email

[guiding_knight]

Privacy concerns aside, if the google technique was applied to emails in the same manner, spam and pornography would be more prominent than any relevant info on many search pages. The sheer volume of this would tip google-style search results. I'm sure the spammers would love this, sending extra, no cost(to them) copies of spam to everyone at the NSA :)

Re:Wrong about email

[Karellen]

I'm sure that UBE would be easily identifiable by a google type of database as practically no mail will exist that goes _back_ to the source.

Filters based on that (to either look for UCE, or to discard it) would probably be trivial based on ratios of sent/received messages to/from a particular envelope.

Score 0, obvious

[AndyChrist]

Most of the article is just stating the obvious.

That last bit about our shadowy overlords, though, that's interesting, and probably the only insightful bit. Although I wouldn't mind a better explaination of why they must use an alta-vista-ish approach.

Re:Score 0, obvious

[Anonymous Coward]

Roger that. Would these people stop just jumping on the I-LOVE-GOOGLE bandwagon and write something worth reading?

Re:How to abuse Scientology

[Anonymous Coward]

... unless of course you're actually a Scientologist, and care about the protection of your religion in order that it remain in its current form for generations to come, untainted by people who simply don't get it and would do very, very bad things to it if it weren't for the protection provided by the various Scientology trademarks.

Posting anonymously, because I am a Scientologist, and that's a dangerous thing to be on /. ... even though I've been a highly active member of /. since it was Chips and Dips ...

Wrong panopticon

[dallen]

Doctorow's point, I believe, is that we have a luxury of choices for searching information, but those who want to wiretap us do not have the luxury of infinite time and infinitely improved ways to find the information they want.

If they could only track us via the public internet, I would probably agree.

I would say we don't know what sort of technology they ultimately have for searching our data; until we knew that, we should not assume anything such as he has, that they're not able to keep up with the flood of data.

Remember that they're not only recording elements of email, phone, and other communications; but they are also tracking who is sending and receiving it; and those who are under "wiretap" are nearly perfectly trackable as long as they can associate an identity to an IP to a person. That is the Panopticon, the prison with ideal survailance; mapping a person to their communication and selectively watching those who bear suspicion.

Sad simplification of storage issues

[wiresquire]

Media type (disk, tape, etc) and size of data is the least of the issues with storing data for long periods of time.

Think of the Library of Congress who want to be able to store data forever. Let's think just 50 years from now. Even if they had the appropriate hardware, do you think they would have a copy of Microsoft Word 2000 handy? MS sure as hell won't be for sale and won't be supported. Would it run on any of the hardware available in 2052?

"Oh yeah. There was this guy called Shakespeare who was supposed to be pretty good, but we just can't get to any of his works anymore".

And ASCII?? That's (largely) fine for English/European, but there are other languages out there that can't be represented in ASCII at all.

Re:Sad simplification of storage issues

[AdamBa]

Think of the Library of Congress who want to be able to store data forever. Let's think just 50 years from now. Even if they had the appropriate hardware, do you think they would have a copy of Microsoft Word 2000 handy? MS sure as hell won't be for sale and won't be supported. Would it run on any of the hardware available in 2052?

Exactly...that's why we need open data formats for everyone.

- adam

Christianity

[Per Abrahamsen]

A search for Christianity gives zero hits for anti-Christian sites. Is that rigged too?

Same for Islam.

Re:Christianity

[dbazile]

I doubt that's the result of an intentional block. It's probably because the search engine assumes that you wont want anything anti- to what you're looking for.

Re:Christianity

[Anonymous Coward]

it's because with christianity the ratio of christianity to anti-christianity is probably extremely high, while with the scientology cult, the ratio is probably extremely low. There are hundreds of websites debunking scientology, but many, many fewer (relatively) debunking christianity.

Re:How to abuse Scientology

[Anonymous Coward]

The reason you should post anonymously is not because you are a Scientologist, but because you're fucking gullible and stupid enough to BELIEVE in it. What a maroon!

Why don't you just kill yourself, fuckwad? I hear there will be a spaceship waiting for you in the next life! Don't forget to wear purple though!

Maybe the semantic web will...

[wiresquire]

That is a very interesting point. If you check out the Semantic Web activity there is a move to semantic definitions . DAML + OIL and several other efforts are all looking at defining the spoken/written language for computers.

I wonder if the number 1 ranked page will always end up being a single document - the ontology.?

incredibly short-term viewpoint

[AdamBa]

1) Google sucks. All search engines suck right now. Altavista may suck 99% and Google may only suck 97%, but they are all terrible, and will remain so until they can actually start to understand what a page is about. The author may bag on AI, and it it bad now, but it's the only hope for workable search engines in the future.

2) What is this absolute crapola about how bytes are more reliable than allegedly "fragile" books? Does this tubesteak realize that there are 500 year old books that are completely legible, while 15-year-old electronic data is unreadable? Yeesh. The only bright spot is that this guy's ravings are in electronic form, so future generations won't have to worry about them.

- adam

Re:incredibly short-term viewpoint

[wwwgregcom]

I dissagree google does not suck 97%, it deffinatly finds what I want very often. I suggest you give google another chance.

Re:incredibly short-term viewpoint

[theNeophile]

The author may bag on AI, and it it bad now, but it's the only hope for workable search engines in the future.

He doesn't exactly bag on AI, he just says we should let computers do what they're good at (Repetitive counting and sifting through masses of information) and let humans do what they're good at (Making judgments on how good or useful a web site is).

2) What is this absolute crapola about how bytes are more reliable than allegedly "fragile" books? Does this tubesteak realize that there are 500 year old books that are completely legible, while 15-year-old electronic data is unreadable? Yeesh. The only bright spot is that this guy's ravings are in electronic form, so future generations won't have to worry about them.

Yeah, 500 year old books are readable, if they're kept in vacuum sealed boxes and not touched by human hands. I have copies of books that are falling apart after a couple of years. And if you had read the whole article, you would have read him say "CDs, magnetic tape, flash, and platters all fall apart pretty quickly -- but that's OK, because bytes are not only comparatively tiny ... but they get tinier every year." Yeah, CDs only last about 15 years, but in 10 years you'll be able to fit your 1000 CD library on 1 SuperduperCD. You can easily make exact copies of bytes, but I'd like to see you make those copies given the 1,000,000 books those 1000 CDs can hold.

Re:incredibly short-term viewpoint

[Anonymous Coward]

you've been trolled, dude. don't worry, it happens to the best of us.

Re:How to abuse Scientology

[Anonymous Coward]

Nice troll. I HIGHLY doubt there are any true scientologists that frequent Slashdot. Slashdotters are idiots in a lot of ways, but most are probably not in that way.

All your ass!

[Tackhead]

From the article:
> How much ass does Google kick? All of it.

ALL YOUR ASS ARE KICK BY GOOGLE!

(Oh hell, somebody had to say it.)

Informative but Not Conclusive

[MadFarmAnimalz]

> Then they must use some hybrid approach: human editors and AI

Well, there's the implied assumption here that the people running this surveillance operate with standard hardware, where standard means something google, altavista, lycos, etc. can get their hands on. Sketchy information suggests that they do not; specialised hardware seems to be the order of the day.

Besides, there's a lot of research going on in terms of context recognition, here to name one place.

Googlepiphany

[Anonymous Coward]

Each time we visit Google, it is with held breath. We have seen the bold 1990s freedom of the Internet dwindle into a thousand fragmented pieces where only the strong survive. Advertisements are everywhere, intruding into our mindscape. The ten thousands of images a year we see, advertising everything from Goodyear-on-a-blimp to online gambling protruding out of your Yahoo mail, are all designed upon the principle of mindless repetition.

It is well understood that the more times you see an image, the more likely you are to purchase its related product when you are wandering down the store aisles, wondering what to purchase. You've had the moment when you're standing in front of seven different brands of raisin brans, and you opt for one or another, little calculating that the one you purchased was simply imprinted upon your brain more times in recent advertising.

Google strides like a valiant and noble knight, a Don Quixote on a mission from heaven, to clear the mindscape of all those lurching, fragmented thoughts: "buy me!" "buy me!" "buy me!"

Like a gift from another universe, where things are cleaner, and evaluated by merit rather than popularity, Google presents an elaborate algorithm for sorting websites into fields of clarity. So insightful is their methodology, other larger search engines have bowed to this upstart. Even the mighty Yahoo, the first big engine on the 'net, has Google under the hood. So do a dozen other search engines, and thousands of sites who have turned their proprietary search functions over to the agile Google churner. AltaVista, Lycos, metacrawlers, and a few other great ones keep the American principle of competition solid, yet here we behold the miracle of Google.

We programmers watched Google come from behind, for we needed a relevance-based engine long before anyone else did: we had to have it so we could put it in the hands of others who needed our services; we were developers: we knew the information was out there, and were willing to spend hours tracking it down. Somewhere along the way, we'd stumble across this small search engine called Google, and discover that it turned up amazingly relevant searches, time and time again. No advertising. Quick.

So we bookmarked it, then we earmarked it, and finally we began to deliver the most precious kind of advertising which can be earned: we told our friends about it. And we delighted in the lack of advertising. Truly a geek's machine; sleek and relevant.

We watched the Internet bubble come crashing down around its own self- exuberance; we all know at least one programmer humbled by the rapid withdrawal of venture capital.

And so we watch Google carefully now, knowing that it is still bearing fruit for its venture capital investors, yet also knowing that our economy is continuing to draw inward, and as carefully as we form our sentences regarding the future of our welfare... we hold our breath when we visit Google each day for its wealth of free, friendly, and advertising-free three billion interrelated facets of information.

We watched Google handle the September 11 tragedy, worried that it might spark them into becoming a news portal, since their cache ability made them compete with sites like CNN which were swamped with 50,000 hits per second... and we saw Google come out cleanly, building on the crisis in a noble, not-capitalizing-on-the-crisis, manner. Now you can visit Google and find current information; it's a portal, yet ever so quietly, since there are no advertisements. Portals have become synonymous with a barrage of advertising, so what do we call this gallant creature who will not stoop to capitalism?

It's just a humble search engine: A search engine which points the way into a future with a clean mindscape. We may not all make it there; spammers prove that they'll come into such a future kicking and screaming for attention, and since we know that we all have to arrive together or else we none of us can arrive, we tolerate them.

Yes, we hold our breath each time we visit Google, lest they make that sad plunge into our noisy world instead of rising above it. And we are continually surprised by the improvements which they are making. These are not trivial improvements, simple cosmetic additions; one by one they have expanded our notion of how powerful a search engine can be, how it can nimbly reach into the deepest crevices of the Internet and produce a slew of relevant information on obscure topics. Search within groups. Search for images. Search only for images which are wallpaper sized from sites in Europe and are black and white.

The essence of the Internet, the information revolution, has somehow been bestowed upon the novel minds working for Google. We look at their job offerings, and yearn for the day when we can deserve such benevolence as to work for Google. Certainly only the best of the best work for Google (or id). They play hockey in their parking lots, and eat catered food every day. Ah, there we begin holding our breath. We like to have fun at work, but too much fun is a sign of venture capital.

How do they do it, how do they keep going, and going, and going without losing integrity by selling ads or trying to do too much? Google quietly inspires us to consider a world without advertising. Oh, they take advertising alright, yet look at it: it's extremely targeted, intended to be relevant to the searcher. With a thick black line separating advertising and content. No advertiser images. None of this irrelevant barrage. Looking for a new ISP? Here's twenty links, and over here in the corner, ten folks who've paid us money to be listed when you search for ISPs. Google drew a distinct line between the advertiser content and their own content. And they steadfastly looked toward our needs when they tolerated no images. Text- based. Get the information into the hand of the gentleman while he needs it, and trust that he will come back later with a thank-you note in hand.

Well, here is one thank you note. I hold my breath each time I visit Google, and I use it extensively, and have for years. I was Googling when Google wasn't yet cool, and I'm delighted to see it surviving. I hope they remain solid in their condition of accepting no image-based advertisements, and pray they will continue to inspire us with clarity on the concept of what it means to serve.

The cache concept, now firmly entrenched in the way we conceive of the Internet, is perhaps the greatest aspect of the information revolution: You once published a site, but now it is defunct. Or your site is presently being slashdotted or DOS'd. No problem, visit the Google cache for the site, and there's your info, as clear and sometimes quicker than the original version. The folks at archive.org have taken this idea and run with it, yet I must admit the first time I realized how profoundly differently we were going to be processing information in the future came when I understood what Google was doing with their cache. I prayed then, and the prayer was answered, that the cache would not be shut down because of re-publishing rights issues. Now Google has enough momentum that it would take an act of Congress to shut off their caching.

Take a look at Google. Unlike most companies with bold pretty mission statements hiding inner corruption, Google somehow matches their ten operating principles with immediate proof. They do it right; they work hard for their money.

Even better...

[Anonymous Coward]

I appreciate Google's integrity in keeping ads unobtrusive, too.

But what I like even better is how good it is at returning useful info. Nearly every day, I pinch myself to make sure I'm not dreaming. Yes, I really do have a reference library that spans most of human knowledge, and yes, I really can look things up nearly as fast as I think of the questions.

I boldly predict that within a decade, it'll be common knowledge that Google has made the human race smarter. Within a decade, schools will grapple with the question of whether students can use Google during tests, just as they grappled with the pocket calculator question a few years ago.

The only thing that would be better would be an open source, massively distributed Google.

Teoma

[cameldrv]

Teoma often provides better results than google.

Don't Panic! (Not online anyway.)

[Bobzibub]

This article is insightful? It is deceiving. I read something interesting about the "Panopticon" not long ago...

"The agency which Poindexter will run is called the Information Awareness Office. You want to know what that is? Think, Big Brother is Watching You. IAO will supply federal officials with 'instant' analysis on what is being written on email and said on phones all over the US. Domestic espionage."
--John Sutherland of UK's Guardian.

Remember John Poindexter? Mr. Iran-Contra? He lied to Congress and kept Ronald out of the loop. He also was responsible for shredding lots of docs on the subject as well. Now he'll be spying on US domestic electronic transmissions.

There is some irony in him destroying thousands emails to cover his ass then and now being in charge of watching everyone else's emails.

I'm also sure that the billions of dollars for his new office may be able to overcome shortcomings of certain search engines. Nobody's going to have to type all those boolean operators.

The quote above is from the UK's Guardian... Check out what you might have been missing

An interesting story, curiously not in CNN..

Nor MSNBC...

Couldn't find it in Washington Post..

Article in LA times on his appointment does not describe what he is to do in his new job except to blather about Sputnik and stealth aircraft.

Not in CBC.ca : (

Cheers to all the spooks! I think it is a job well done! -b.

Re:How to abuse Scientology

[Anonymous Coward]

Lick my shiny OT nuts, mortal.

Google: Big improvement, but not perfect

[livingdots]

I like Google; it weeds out most of the spam -- unlike AltaVista. It isn't perfect, though. I once searched for prostate milking, after reading this. The search results were quite interesting: It brought up hundreds of, apparently fake, headlines ("Located here! Prostate Milking") and domain names ("childhood-disease.accurate-health.com/prostate-m ilking.html"); it in fact still does, even though a month has passed since. Many of the links don't work, but some redirect you to other sites (this one amazingly owned by Novartis, a supposedly "respectable" biotechnology company). Question: How do they do this?

The /. contradiction in one sentence

[Anonymous Coward]

"I hate it how everything is being cached and observed and indexed, but I love it cuz its cool!!!"

Something you might not know about Google...

[hlh_nospam]

The folks that run Google(tm) have decided to censor advertising of perfectly legal articles, and to enforcing a ban on advertising by anyone who sells knives, firearms, or related items, whether or not those items are featured in the actual advertisement. See
this link for details.

Re:Something you might not know about Google...

[Gonzoman]

And this is a bad thing?

Re:Something you might not know about Google...

[stux]

The Google search engine company, refuses to advertise businesses related to the gun or knife industry

Why is this news? Because gun and knife owners are being discriminated against. Just imagine a storeowner who posts a sign on his door saying "No Firearms Allowed". While still open to the public, anybody can walk in and shop for products. However, the store owner is saying to you, the gun owner, that while your business and your money is welcome here, your firearms are not.

Oh I love it.

[BEGIN BROADSWEEPING GENERALIZATION]
BTW, in the real world, this is what most people would EXPECT.

Perhaps the realworld doesn't quite extend to the US, I don't know.

BUT in the RoTW a shop owner can easily decide to put up a sign which says "No Firearms Allowed" and expect it to be respected... But what's more, they don't have to... BECAUSE THAT'S THE DEFAULT!

So, I have no problem with the shop owner deciding to ban firearms... its 'his shop' he can do whatever he wants, if you don't like it, don't give him your money. Your loss.

Meanwhile, I'm quite pleased that Google refuses to accept money to host gun and gun part ads.

Go Google :)

Gun Owners can be a funny bunch... this one uses some very nasty, angry, retalitory and confrontational language...

Funny Americans.
[END BROADSWEEPING GENERALIZATION]

Re:Something you might not know about Google...

[isbhod]

So, I have no problem with the shop owner deciding to ban firearms... its 'his shop' he can do whatever he wants, if you don't like it, don't give him your money. Your loss interesting... so then by your logic a store own could but up a sign that says "whites only" or "no darkies allowed" or "only crackers can shop here" or..well i hope you get the point. And he can get away with this because, as you say, "it's his store"? you may say that discrimination of this kind is different because with the "no gun sign" the owner is discriminating against an object, not a human as with the "honkey" signs. And I'll give you that, so then lets change the "whitey" signs to "no prosetic limbs". Would you then say "it's his store, he can do what ever he wants."?

And the moon landing was faked, too.....

[xmark]

The notion that Echelon hasn't been proved to exist is both uninformed and naive.

Cryptome.org has a definitive collection of documents concerning Echelon in an archive. Those desiring to test directly for themselves the existence of Echelon might consider sending some email using phrases from the Echelon trigger words list. This list, by the way, was circulated last year on newspaper wire services and isn't exactly top secret.

Good!

[Tim Ward]

So Google is even better than I thought it was!

Slashdot headline: Easter Egg found in Human DNA

[diamond0]

Heck I am still waiting for folks to find a licensing and copyright statement in the human genome.

Is anyone looking?

Seriously: we can get the raw data, right? Has there been any concerted effort to find any meaning in DNA at all other than the blueprint for life? We've known about mother nature's most reliable data store for decades, now. Are we sure, yet, that the complete works of the great society of 10^n years ago are not just waiting to be found?

Gov. search technology

[xiitone]

Haven't seen a link to this yet. The CIA is funding new search technologies via In-Q-Tel From their page:

In-Q-Tel is an independant, private, non-profit company funded by the U.S government with one objective:to identify and deliver next generation information technologies to support CIA's critical intelligence missions.

I wonder if they like soda?(Hi Cory!)

Re:But other times you should USE google!

[jeffehobbs]

Google's got a great spell check...

"Did you mean: brilliant "

I apologize in advance; if it was any other word, I wouldn't have made this snarky post.

~jeff

Death of google imminent; Film at 11. NOT

[limbop]

as i mentioned in a previous post, Google ranks pages in a recursive way. An important page is one that is pointed to by a lot of other important pages. so the flaws in your argument are:
1. Your page is probably not all that important. unless you actually have important information on your page which most web pages dont.
2. If you dont put any links on it then it becomes even less important, because now it is not even a hub.
and to a lesser extent
3. what you are suggesting is against the nature of the web in general. web page authors dont supply links only because it is hard to find links otherwise. they supply links as part of the text, for example when quoting or they supply links because they think those specific links are important and not others etc.

in short the scenario you describe is both unlikely and not as catastrophic as you think it is.

limbo.

Re:Death of google imminent; Film at 11. NOT

[Tim Ward]

Just a couple of points:

(1) My page is important to a lot of people as it does have important information which is not available anywhere else.

(3) People are supplying fewer links already in email. How often do you email someone a long complicated URL these days, and how often do you now email them "Google for xxx yyy / I'm Feeling Lucky" - quicker and easier to type and read? I haven't seen many Google search strings replacing links directly on web pages yet, but who knows?

Windows XP

[Rareul]

I have a couple of interesting comments regarding searching and XP:

1) TweakUI, part of the XP Powertoys released, then later unreleased, has a parser for IE. It enables me to search from the Address bar using only a single letter to designate where I want to search. Thus, when I want to search google I type: "g [insert search terms]". Here are some of the URLs, (these should NOT be hyperlinks):
d - http://www.dictionary.com/cgi-bin/dict.pl?term=%s
g - http://www.google.com/search?q=%s
t - http://www.thesaurus.com/cgi-bin/search?config=rog et&words=%s
y - http://search.yahoo.com/bin/search?p=%s

2) Whenever I screw up typing in the address bar (i.e., whenever I forget to type the 'g' or 'd'), an MSN search page gets pulled up. Of course you can disable this searching from the address bar in the options menu. But if you screw up typing again, the option automatically turns on and pushes you further into M$-land. IE 6.0
sp?

Re:Windows XP

[Anonymous Coward]

FYI, the letter search terms have been their since IE3 at least, and the Powertoy thing is just a GUI frontend.

Check under HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\SearchUrl to edit (and alter the MSN default).

Google is good for that too !

[Anonymous Coward]

I have the Google scout button in my personnal toolbar and when I'm "wandering" around the web and find a interesting/funny/brilliant site, I often Google-scout it to go to other interesting/funny/brilliant pages.

why cant altavistas and yahoos copy google's appro

[Tablizer]

Is link/reference indexing an exclusive patent of Google? It is a bit hard to believe that they are the first to do this. Books have had references in them for mellenia. Nobody bothered to EVER index those until Google? (And write about it.) I find that a little hard to believe.

The last thing we need is an e-nopoly on search engines.

Link interconnections in Google

[Steven_Wostoen]

The basic design of the Google cluster unfortunately lends itself to this kind of exlusion in the linking moreso than other search engines or entities containing linking mechanisms, but, this is not neccessarily a bad thing.

The cluster receives the client request and reverse-NATs a reply based on an advanced TLU setting, which weighs variables against cached requests linked to the hashed lists of previous search requests items and returns. The problem comes in when each node of the cluster contests the cache servers for permission to send info back to the python code in the back-end web server.

Often, permission is given to two nodes on the server or more, and this causes a problem in that the same info is sent over and over, causing linking problems after the python code is processed and spits out the HTML to the front end web server. This was the only way to do it and still keep Google's unique search features.

Uhm, what is insightful about this article?

[Anonymous Coward]

...why can't you link to someone who has something interesting to say about search engine technology instead of some simpleton raving on about things the rest of the world has known about for YEARS now...

THIS IS NOT INTERESTING. IT IS NOT EVEN WELL WRITTEN.

learning to love me?

[panopticon]

For a second, I thought it was an article about loving me. I know I can be difficult and high-maintenance, but it can be done, I swear.

Minor correction

[Binary Tree]

The cluster receives the client request and reverse-NATs a reply based on an advanced TLU setting, which weighs variables against cached requests linked to the hashed lists of previous search requests items and returns. The problem comes in when each node of the cluster contests the cache servers for permission to send info back to the python code in the back-end web server.

This is slightly inaccurate and misleading. The truth is that the 4-way database clusters an array of search requests based on a dynamic SQL query.

Just a head's up.

Is this the OpenCola guy?

[Anonymous Coward]

Haven't heard much about Opencola lately. I'm assuming this is the same guy. Since OpenCola was supposed to be able to do some searching and evaluating I always thought it would be competing with Google at some time in the future.

Unimpressive argument.

[fm6]

What does this article show, if anything? That a simple search for "scientology" is dominated by official CoS sites? So what? That would be true for any organization, never mind one that owns a zillion different domain names. You still break out if you add even a single keyword. Or if you skip the first few pages of results.

It seems to me that those 50 or so "official" hits are not a result of a deliberate attempt to dominate Google results. They're just a symptom of the way Scientologists -- like any other religious zealots -- love to blather about themselves.

Another problem... Google Spyware now in use!

[ip4noman]

A change has just happened at Google: they are now tracking all off-site links (they used to only track off site links to advertisers). Where you used to get a link like this:

http://www.some.site.com/foo/bar

You now get a link like this:

http://www.google.com/url?sa=U&start=3&q= http://www.some.site.com/foo/bar &e=code

Now they *could* be (and knowing google, probably are) using this to improve the quality of searches, by watching how many links a person takes on a specific query, and assume when they stop, they found what they were looking for, and rate the followed links higher next time a similar search occurs

But perhaps something more sinister is at work. This information could be of great value to direct marketers and police agencies. Google now not only knows your IP address and your browser type, they now know where you are going.

Arguably, Google has the highest quality search results, and they have operated for at least 2 years without advertisements, solely on venture capital (and it must have cost a fortune for the hardware, and all those PhDs). Now they have us all hooked, they begin tracking our movements.

Makes you wonder where all that startup funding came from and what revenue sources will contribute to the payback...

Re:Another problem... Google Spyware now in use!

[ip4noman]

Huh? Now links to offsite locations are normal again. Looks like something they were playing with (at Mar 11 2002, 0058 UTC, restored by 0100 UTC)

Re:Another problem... Google Spyware now in use!

[PurpleFloyd]

Google has had ads for years. They have been off to the side in little pastel-tinted boxes. The great thing is you never even notice them until one of them is useful -- unlike all the damn popups everywhere, they stay out of the way until you need them. As such, I will willingly click on a Google ad if it relates to what I want (and it usually does!), while popups are killed via Mozilla, or, failing that, immediately destroyed as soon as they come up. Interesting to note that while everyone else seems to take the idea of "ads should obscure content", Google has taken the rational and sane approach of "ads should be relevant content".

Re:Another problem... Google Spyware now in use!

[Anonymous Coward]

It seems to happen on and off for me.

"NEAR" operator

[eples]

The grim era before Google, when searching was a spew of boolean mumbo-jumbo, NEAR this, NOT that, AND the other?

I kind of liked the "NEAR" operator - wish google had it!

alleged fragility of books

[AdamBa]

Maybe 500 was an exaggeration (given that the printing press was about that old)...but there are certainly 300 year-old books that are fine (not having been vacuum-sealed) and 100 year-old books are not even that unusual.

The article (or that part of it) reminds me of the people who claimed that newspapers were going to fall apart and they all needed to be microfilmed and stored that way...now the newspapers that were dumped are in such great shape that The Sharper Image is selling them for $30 a pop, and the microfilms are deteroriating, that is the ones that were made legible to begin with.

Copying bytes may be easy but every time I switch computers I have to worry about moving stuff and where is it stored, then there is 20-year-old stuff on 5 1/4" floppies...meanwhile my books from childhood are all doing great. Even the cheap-o dot-matrix printouts from my BBS days in 1983 are perfectly preserved, which is more than I can say for any data I had from back then.

- adam

Re:alleged fragility of books

[NaturePhotog]

Maybe 500 was an exaggeration (given that the printing press was about that old)...
Actually, there are books that pre-date the printing press. The oldest printed book still around is The Diamond Sutra, at The British Library. It dates from 868AD.

It may also be the oldest existing Open Source document:

The colophon, at the inner end, reads: `Reverently [caused to be] made for universal free distribution by Wang Jie...

:-)

You're missing the point

[Anonymous Coward]

Google is successful only because it uses the collective judgment of all public web authors. This approach won't work at all for private documents, which can't be ranked based on how many links point to them. The government's only option is a pure machine-based intelligence, which so far has not been a very successful approach.

Google Can Search Your Apartment and Your Brain

[weston]

Paul Ford wrote a hilarious piece on what life might be like if google tried to index the world.

Me, I think that the reason that the Harry Potter film ended up looking uncannily like what was in everybody's head is because Google can index the brain.

Just a theory.

Google sometimes defies explanation.....

[fwc]

I was talking to a friend about "mystery email attachments", and wanted to find this user friendly strip.

So, without thinking I fire up google and type the search:

"user friendly the comic strip" email attachment

and then clicked on search. The first hit is the cartoon I wanted, so I click on it. When I pull up the page, I realize that the text words "email attachment" don't appear anywhere on the screen other than the graphic text in the comic itself, so google shouldn't have found the page - at least according to how I thought google worked. So I pulled up the source to see if there was a meta tag there which would explain this. Nope.

The only thing I can think of is that google either OCR's the pictures (seems scary, and that font which Illiad uses doesn't look very OCR-able). The other thing I thought about is that perhaps google also matches text found within <A> tags which link to that page or something.

I've shot a message off to google to ask about this but I haven't heard back yet. I'll be interested to find out how the *@(#*$ they did this.

I think that I saw an ad somewhere which said "How the @(#$* did they do that?" was the highest praise one web designer could give to another. If that's true, they've definately earned my praise in this case. Regardless, some wizard at google got their search engine to do exactly what I wanted with whatever technology they used. Technology sufficiently advanced is indistinguishable from magic. And google is definately magic.

Re:Google sometimes defies explanation.....

[mbauser2]

It's no great mystery. That's Google's link analysis at work. Google associated the words "email" and "attachment" with http://www.userfriendly.org/cartoons/archives/99ju l/19990724.html because other pages linked to it using those words.

For example, this page at whump.com linked to the strip using "how to handle those attachments" as the text of the link. There are presumably other pages with links that contribute to the effect.

Goggle itself admits this is the cached version of that User Friendly page. I actually see this a lot when looking at the cached versions of sites I've found through Google.

it's not just scientologists

[NFW]

Do a google search on 'crucial facts,' skip the first ~5 results, and most of the results beyond that point are just search-spam.

Some dork has registered a bunch of domains and created pages titled "crucial facts about [keywords]" with meta refresh tags to transport you to his/her/its web-based storefront for unrelated trinkets (or just-barely kinda-vaguely-sounds-related trinkets).

I stumbled on this while searching for motorcycle clothing, but judging by the "crucial facts" result set, there's hundreds of these little spammer droppings in the google database, just from this spammer alone.

Search | I'm Feeling Lucky

[hazyshadeofwinter]

I think the "I'm Feeling Lucky" button shouldn't take you to the first result, but rather the n'th, where n is random. Otherwise, luck doesn't really enter into it.

Re:Nice description - Flawed conclusion

[Tungbo]

The author, unfortunately, seems woefully ignorant of the advances made in the field of Natural Language processing. There are already a number of products available which analyzes Wire Service reports semantically to filter out relevant news items. Thus, the fact that e-mail or IM don't have links doesn't mean that they cannot be processed by computers.

While it is true that someone trying to fool the NLP programs can easily do so ( by using coded phrases, etc. ). But human analyzers are not likely fare much better in those cases either.

This is why privacy legislations similar to what's instituted in Eurpoe are still necessary in the US of A.

Cory Doctorow

[HP LoveJet]

...is really cool. And he writes real good. And he understands technology. *And* he's an Award-Winning Science Fiction Writer!

Hey Taco, can we replace JonKatz with Cory?