Google De-indexes Talk.Origins, Won't Say Why UPDATED

J. J. Ramsey writes "Talk.Origins is an archive with thousands of pages exposing creationist pseudoscience. Rather mysteriously, Google pulled the plug on its search engine, giving only the vague reason: 'No pages from your site are currently included in Google's index due to violations of the webmaster guidelines.' This was apparently triggered by a recent cracking of the site that added 'hidden links to non-topical sites,' but Google won't say just what the violations were. Talk.Origins webmaster Wesley R. Elsberry believes that this Google policy harms honest webmasters." From the article: "My mission, whether I liked it or not, was to find and fix whatever problem the [Talk.Origins Archive] might have, with no guidance as to what the problem was and nothing at all about where to start looking... I was extremely lucky. The damage to my site was limited and in the first place that I happened to look. Other honest webmasters might not be so lucky. They may have to undertake an arduous process of vetting pages, essentially having to second-guess the mind of the cracker in trying to locate a problem that Google knows the exact location of." Thanks to an alert reader who sent in Matt's blog posting about how Google handles hacked sites.

Comments

huh?

[Average_Joe_Sixpack]

What's this?

Re:huh?

[Daniel Dvorkin]

That's the Google Groups archive of the talk.origins newsgroup, which is a different animal (an ancestral form, one might say) from the Talk.Origins Archive web site. It was the site that was delisted.

And indeed, as of right now (10:35 PM CST) a Google search for "talk.origins" doesn't show any links at all to the Talk.Origins Archive. In fact, the first link that comes up is to a young-Earth creationist site which claims to offer "intellectually honest responses to the claims of evolutionism's proponents, including--but not limited to--the 'Talk.Origins' newsgroup and the 'Talk.Origins Archive' website."

Conclusions about species competing in crowded niches are left as an exercise to the reader.

Re:huh?

[bytesex]

evolutionism, homosexualism - they're all new, and evil ! Better stick to the good old isms such as religionism, wifebeatism and alcoholism.

Re:Words are Meaningless

[Baricom]

Nobody was evil here. The guy's site got hacked and spam links added, Google rightfully de-listed him, and then the webmaster found the problem, fixed it, and asked Google to re-list. Am I missing something?

Google censoring Usenet? Not!

[BorgCopyeditor]

The writeup sucks. It implies that Google is censoring Usenet.

The problem

[Aexia]

was that he had no idea why he was delisted so he could fix the problem.

Re:The problem

[wfWebber]

And with good reason I'd say. If I add a couple of ways to "fool" a search engine to my web pages, I can't seriously expect that same search engine to tell me which of the tricks they discovered?

Re:The problem

[DrHyde]

Surely the problem is that he didn't run his site properly (ie, random strangers could update it) and that when they had done that he had no easy way of finding damage they might have done throughout the site. He moans about having to check 5000 pages by hand, but I sure as hell wouldn't have to manually check the 7000-odd files that make up my site.

Re:The problem

[NoTheory]

Er... People seem to have some misconceptions about Talk.Origins. The point of Talk.Origins is to catalogue the arguments that Creationists make, and the appropriate counter argument to any given creationist argument. This is primarily due to the fact that creationist arguments are extremely unoriginal and repetitive (seriously, creationists have been hauling out the same arguments since before Darwin's time, regardless of the change in scientific thought or theory). So i'd say that Talk.Origins does a very good job of presenting both sides. It just happens that the creationist side is out-dated and unsupportable. But that's a matter of fact, not of inaccurate representation.

Also, we can't accurately ascertain whether the earth is the center of the universe or not. It's probably not, but the way space time expands gives no reference point for the point of origin. From any point in the universe, it looks like everything is expanding away from you.

But you are correct, the earth is indeed not flat. It's sort of a squashed sphere.

Re:ahhh i love it

[scowling]

Except, of course, that "creationist" does not equal "Christian". Talk.origins exposes *all* creationist pseudoscience, from *all* sources.

You love to whine, don't you?

[BorgCopyeditor]

"exposing creationist pseudoscience"...

Slashdot is so biased I don't know why I even bother anymore. Bashing Christians is so fashionable these days.

"Creationist" != "Christian", but don't let that stand in the way of your pretending to feel victimized.

Re:You love to whine, don't you?

[BitchKapoor]

Is that what the Bible says? No, it isn't. The Bible says Jesus created a perfect, sinless world. Man sinned, and brought the curse of sin and death on mankind -- man's choice, man's free-will.

This seems contradictory to me. Man is part of the world. How does your definition of perfect and sinless make any sense, if man (as part of the world) is able to introduce sin? That's like saying Windows '95 is stable just because it hasn't crashed yet, even though it could if you run an ill-behaved program. Or a Unix machine that's on the internet with remote logins available for root, with no root password is secure, just because no one's logged in and done anything malicious yet.

But enter what Jesus did, its anything but. Its grace, compassion, and mercy beyond anything ever known.

So he made the world, thought it was secure, then man hacked in, jesus realized he fucked it up the first time, so he released a fix? And you worship this dude as being the best? Oh, plus then there's all those little undocumented patches that the church slipstreamed in over the years. Sounds like the christian influence is what ruined the software industry!

Re:ahhh i love it

[One Louder]

"exposing creationist pseudoscience"... Slashdot is so biased I don't know why I even bother anymore. Bashing Christians is so fashionable these days.

Wait a second - I thought that creationism was a "valid alternative scientific explanation for the origin of the species", and not religion. Are you saying that it's really religion, specifically Christianity , wrapped in deceptive packaging?

Sounds like you blew the cover there, dude.

Whine, Whine, Whine

[MDMurphy]

So many people refer to Google as if it were a human looking at web sites and giving it the big thumbs up or down. As part of the indexing if the spider finds "violations" such as presenting a different page to spiders than to humans, it risks being dropped from the index. To expect a human response to why each site triggered the de-indexing is not reasonable.

In the webmaster's whining about Google, he complains about the request to be re-indexed containing:

                    *I believe this site has violated Googles quality guidelines in the past.

                    * This site no longer violates Googles quality guidelines.

He thinks these are "an admission of guilt", but they dont' say "I violated" they say "the site violated". So, if the site were hacked and did violate their indexing policy, fix it, say you've fixed it and move on. How many hits has he had over the years that came directly from Google? And did they come from Google due to all those people choosing Google to search for his site or it's topics? But now he whines about being delisted for the time it takes him to fix a site he should have kept unhacked in the first place.

Re:Whine, Whine, Whine

[identity0]

Heresy! Google sees all, Google knows all! Google is a man, a spider, and the holy page in one!

Brin 3:14 "And Google so loved the internet, that he sent his only-born son Larry Page to it so that any who believe in him shall not perish but have ever-lasting life in the Googleplex."

So you see, there *is* a person, Larry Page, who is also the spider that indexes everything and is also the page that serves up results. Only through this holy trinity could results as good as Google's result, thus proving Google's divinity. If the almighty Google has delisted this sinner's page, then we should not be looking at it in the first place, yes? To go against the wishes of Google brings hellfire!

Re:Words are Meaningless - Public Utility

[TubeSteak]

Like it or not, Google has essentially become a Public Utility.

I'm going to have to go ahead and disagree with you on that.

People may be treating Google as a public utility, but Google (a private company) has absolutely no obligations to any website.

To just pull the plug because you somehow -- maybe not even your fault -- ran afoul of a constantly changing set of rules is not aboveboard behavior for a $157B company.

Ultimately, Google* has the right to change the rules when & if they please, in an arbitrary fashion, without consulting anyone.

*When I say "Google" I mean "the guys who own a majority stake in the company and cannot be overruled"

Re:Words are Meaningless - Public Utility

[vixen337]

I was under the impression that they told the webmaster the reason they were delisted, they just didn't tell the webmaster the specific pages that the reason pertained to. Like "Your site has been delisted for hidden links to non-topical sites" instead of "Your site has been delisted for hidden links to non-topical sites on pages index.html, intro.html." etc. To me, that's a webmaster job. Google did their job on their end. What if the site had hundreds of pages of non-topical links? What if Google spiders just stopped at the first one they indexed (as they should). Should google be in charge of going through this guy's site and telling him exactly where the problems are? They are a search engine, not a website security firm. People are getting lazier everyday and everyone expects someone else to do their dirty work for them. People need to take some responsibility and stop whining.

Re:Words are Meaningless - Public Utility

[telbij]

Unfortunately you're missing something too.

Google is in an arms race with spammers and blackhat seo firms. How are they supposed to know whether someone is honest or just mining them for information for their scam?

Synopsis

[operagost]

"Talk.Origins is an archive with thousands of pages exposing creationist pseudoscience"

This article is a submission containing a biased summary which has little to do with the actual topic, which is the enigmatic status of Google's search algorithms.

Re:Synopsis

[TapeCutter]

To paraphrase your post: "science is a religion". I disagree, please correct me if I misunderstood.

Science is based on a single "article of faith", ie: I belive the real world exists as I and others who inhabit said "real world" collectively percieve it. I can only prove with certainty that I exist and furthermore can only prove it to myself. If I do not have faith in the real world then "others" must be a figment of my imagination, a troublesome state of affairs since the imaginary/real others will declare me a psychopath and lock me up in a real/imaginary padded room.

Since I and "others" can observe and agree on things in and about the real world we can create testable theories that can be refined to better fit our observations and accurately predict outcomes. ie: We can practice the scientific method and refine our theories until we reach a (possibly non-existant) point where the only "assumption" is that the real world exists, or as I like to put it the Universe "just is".

So regarding a belief in evolution - The only faith required is the faith that the real world exists.

As for religion, it is based on blind faith, blind since I and "others" cannot percieve the same observations, these observations and associated theories fail the "real world" test because they cannot be demonstrated to "others" using their own perceptions. This does not mean religion is pointless or even psuedoscience, it simply means religion is not comprable to science (apples vs oranges). In my mind making such comparisons entirely misses the point of both endevours.

Psuedoscience, litteraly "fake science" is blind faith dressed in a lab coat. Sure creationism is a theory but it's NOT a scientific one, claiming otherwise is by definition, psudoscience.

Finally the lack of a strong scientific theory for the origin of life does not validate creationisim, nor does it invalidate the theory of evolution.

Bias: I suppose you could argue on some deep philosophical level that faith in the "real world" makes me biased toward...um...the real world, I can only wonder if that automatically means psychopath are unbiased? What does "science is a religion" prove? - I'm biased because science has a demonstratably superior track record of explaining and predicting the real world's behaviour whereas blind faith performs no better than random chance. Is that the kind of "bias" we are talking about here? - Because if it is, I am wondering how a non-phycopath can go to bed confident they will awake on the same planet the next morning?

Short cut to scientific enlightenment: Carl Sagan's book "A demon haunted world". It's also serves as an outstanding example of what a skeptic should be.

Re:Synopsis

[Tyreth]

I think that the person you are replying to is referring to the big bang. If you look far back enough into the history of the universe, you get to a point where everything began to exist. At the singularity of the big bang, we find that both time and space began. There is no "before" the big bang, as time did not exist. This is a central part of the cosmological argument for God's existence:
1. Everything that begins to exist has a cause
2. The universe began to exist
3. Therefore, the universe has a cause
What came before the big bang? That question is meaningless, as time did not exist. So you have a few options, only one of them feasible. The first is that the universe is infinitely old and had no beginning. Once a view of atheists, this is no longer scientifically plausible. The second answer is that the universe came into existence from nothing - absolutely nothing. The third, and most reasonable, is that something else caused the universe to be created. This cause must itself be timeless, and spaceless, as time and space began to exist with the big bang.

So the atheist must either claim the absurdity that the universe came from nothing, or he(/she) must acknowledge that there was something that created it. And that *something* is inaccessible from scientific analysis. It is not, however, too far from the reach of philosophy and logic. We can draw reasonable conclusions about this entity.

Re:Synopsis

[AlXtreme]

What came before the big bang? That question is meaningless, as time did not exist. So you have a few options, only one of them feasible. The first is that the universe is infinitely old and had no beginning. Once a view of atheists, this is no longer scientifically plausible.

Sorry, you are incorrect. There wasn't any time before the big bang, thus nothing could have come before it as you yourself stated. Not even time, as the big bang was the beginning of time. However one alternative view is that the universe oscillates between a big bang and a big crunch, thus from a timeless point-of-view the universe would be infinite.

What I really find worrying (hello, 1500's are calling) is the method of reasoning by creationists, like yourself.
A: There was a big bang.
B: We currently don't know what was the cause of this.
C: There must be 'some higher being' that created the universe.

Now A and B do not lead to C, no matter how you reason. If you want to have a drop of credibility, you'll have to support your claims. However, you can not, thus your logic is flawed. What created the 'entity' you speak of? What came before it? Why did it create the universe? If you want to play the science game, you should be answering those questions. Science allows questions to be left open, but tries to answer as many as possible by using facts. Creationism is not, and is unlikely to ever be, scientific or logical. You are allowed to believe in the toothfairy for all I care, but unless you have evidence that a mystical entity is willing to pay for your teeth: keep your belief to yourself.

Re:Words are Meaningless - Public Utility

[Anonymous Coward]

Public utilities for a town have certain responsibilities only because they have accepted those responsibilities in exchange for the town making them a monopoly.

Google has no such responsibilities just becuse of the way they're treated by users. (And even if you argue that they're a monopoly, they haven't been granted monopoly status by a government.)

Re:Words are Meaningless - Public Utility

[eclectro]

What you're missing is that Google gave him no clue/hint/guide/comment/help on why he was delisted.

I'm not for censoring any information, and I am not trying to defend google. But there may be one very good reason why this may be happenning this way.

Google is at war with search engine spammers. When google de-lists somebody for spamming their search engine, if they gave a specific reason why then all the spammers would do is tweak their spam farm and be up and running in a couple of hours.

If they told this guy what was wrong, they would have to spend a huge amount of time and resources telling why everyone is wrong, all the while helping out the spammers.

Google is a good search engine, but if you notice that if you go beyond a couple of pages out of search results, many times you will find nothing but useless "link farms." Unfortunately, spam is no longer limited to email inboxes anymore, it's everywhere.

Understandably confused that some is not all

[Nevyn]

They may have to undertake an arduous process of vetting pages, essentially having to second-guess the mind of the cracker in trying to locate a problem that Google knows the exact location of.

Bzzt. The website admin needs to locate one or more problems (== however many the cracker planted), and Google knows the exact location of at least one. "one or more" >= "at least one". If google tells people where their problems are, google will be playing whack a mole for eternity. There are contractors/services that should be able to help them/anyone, google is not one of them.

Caped Hacker

[derubergeek]

This was quite obviously the work of the Flying Spaghetti Monster.

probably just bad algorithms

[martin-boundary]

While it's natural to sympathise with the victimized website, it doesn't follow that Google is doing something Evil(TM) in this instance, rather it's most likely that their current algorithms are badly tuned.

With the index sizes that are being collected by search engines these days (on the order of 10 billion entries), it's completely naive to think that some humans are sitting at a terminal choosing to delist websites for some policy reason or other. It's also completely naive to think that a human email monkey can do any sort of digging to find out the exact reason that Google's automated algorithm has censored this particular site.

Instead, Google's engineers have automated algorithms which do all the censorship, and the policy is just there as a thin cover for whatever the algorithm happens to be doing today. It's worse of course, because 1) algorithms change every few months and 2) there's simply no comprehensive way to test the quality of the implementation.

Anyone who's programmed a nontrivial algorithm knows that obscure edge cases are a bitch, and with 10 billion websites, any algorithm will have plenty of obscure edge cases which nobody has ever tested, nor ever will. The most likely explanation is that the website in TFA is a false positive of some subsystem, but fixing it will require changes to the algorithms, and Google don't want to risk that, would you? The problem will probably go away in a few months when the algorithms are scheduled to be updated.

Re:probably just bad algorithms

[suv4x4]

While it's natural to sympathise with the victimized website, it doesn't follow that Google is doing something Evil(TM) in this instance

They really trademarked that? Shit, they don't really leave people much choice than being good these days.

Re:ahhh i love it

[thefirelane]

ID does not propose that the creator must be a diety

ha ha ha ha. Yes, in ID the creator must only be someone eternally existing with the ability to manipulate all matter in the universe at will.

But diety [sic].... no!

In case you missed it, in ID it must be a deity, or else who created the creator? If life can not come from non-life, then there must be some eternally existing intelligence to kick things off (aka God). So either you don't understand the theory, or you are lying.

You have to love when a theory tries to sound more sane by saying "but... it could be space aliens too!"

Is there anything I'm missing there about ID?

Re:Words are Meaningless - Public Utility

[TubeSteak]

Google is a Public company, not a private company.

Google is publicly traded, but for all intents and purposes, privately owned by 3 people (who control 66% of the shareholder votes).

So if they know where the problem is, it would be *good* for them to help out and point the site admin at the problem area. Right?

It might be good, but my point is that Google doesn't have to... and maybe shouldn't.

To some extent, part of Google's ability to foil bad website behavior relies on security through obscurity. If Google doesn't tell or hint to anyone how the cheat-detecting algorithms work... well, isn't that good for Google?

I could make the argument that since (as you argued) Google is a public company, they have to do what's best for the shareholders by doing what's best for Google. But that is an irrelevant argument, since there's really only three people whose opinions on the subject matter.

If Google ever did do something along the lines of what you're proposing, they'd have to put a lot of time & effort into setting up a system that can't be easily abused by link spammers, is easy to use for idiots, etc etc etc.

That may be more trouble than it is worth, compared to saying "not our problem, deal with it yourself."

Re:Words are Meaningless - Public Utility

[jrockway]

> Google is at war with search engine spammers. When google de-lists somebody for spamming their search engine, if they gave a specific reason why then all the spammers would do is tweak their spam farm and be up and running in a couple of hours.

Security through obscurity is no security at all. The spammers already know Google's weaknesses -- that's why there's so much spam everywhere.

Re:Words are Meaningless - Public Utility

[coaxial]

People may be treating Google as a public utility, but Google (a private company) has absolutely no obligations to any website.

PG&E is a public company. ComEd is a public company. Verizon is a public company. AT&T is a public company. They're all public utilities. Simply being a publicly traded for profit corporation doesn't mean that you're not a public utility.

Ultimately, Google* has the right to change the rules when & if they please, in an arbitrary fashion, without consulting anyone.

Yes, but there is something called ethics. Google is held to a higher standard than the Ackbar and Jeff's Falafel and Oil Change Hut because of their unique position of being depended on by hunderds of millions of people the worldwide. Also, Google said they should be held to a higher standard with their "Don't be Evil" slogan.

Did Google act wrong in this case? No. But that doesn't mean that your larger point about corporations are beholden to no one is valid.

Re:Words are Meaningless - Public Utility

[beoba]

You're missing the point. The article mentions that the webmaster got this message: No pages from your site are currently included in Google's index due to violations of the webmaster guidelines. Please review our webmaster guidelines and modify your site so that it meets those guidelines. Once your site meets our guidelines, you can request reinclusion and we'll evaluate your site. Which insinuates that there is a blacklist somewhere which contains talkorigins.org. It would not be a big deal to add an additional field to that listing which would allow for the following improved message: No pages from your site are currently included in Google's index due to violations of the webmaster guidelines on http://www.talkorigins.org/index.html (and possibly elsewhere). Please review our webmaster guidelines and modify your site so that it meets those guidelines. Once your site meets our guidelines, you can request reinclusion and we'll evaluate your site. See? Sure, it would be "easier", but a useful feature is still a useful feature, and this is one that would be easy to implement.

Google Webmaster Tools

[RockoW]

Google have a set of http://www.google.com/webmasters/tools/ tools for webmasters. essencially it give out every diagnostic needed to fix your site for Google. Additionaly you have statistics for searches and how GoogleBot see your site. So, you shouldn't blame until you googled for the answer! Searching for "Google index tool" shows up "Google Webmaster Central"...

Re:Google Webmaster Tools

[DrXym]

Google certainly has some useful tools, but when they don't work you are screwed. I have a site which I won't name which is not indexed by google and I have absolutely no idea why. I've submitted the url, built a sitemap using their own tools, validated it and even submitted the site for relisting. It still isn't there. What have I done wrong? The tools say everything is fine except it isn't. I could go to the web forum but other postings suggest the employees will likely just tell me wait for indexing. Except its not indexing me.

The sick thing is that I have Google Adwords on that site so each day that Google don't list me, THEY are losing money. I estimate I get 10x the click through business from MSN search than I do from Google. I'd probably make 3x the profit (as would Google) if they'd index.

Re:Words are Meaningless - Public Utility

[Hoi Polloi]

What if you went through airport security and instead of saying "you can't carry that penknife on board" they just said "you have something forbidden"? When I code an error message into a piece of software I don't just say "You did something wrong" I know what the cause of the error is so I tell them.

Re:Words are Meaningless - Public Utility

[psiclops]

To some extent, part of Google's ability to foil bad website behavior relies on security through obscurity. If Google doesn't tell or hint to anyone how the cheat-detecting algorithms work... well, isn't that good for Google? they dont have to tell anyone how they found the problem, just where. if the webmaster of a site is deliberately trying to cheat google, they already know what pages are in offence anyway.

I could make the argument that since (as you argued) Google is a public company, they have to do what's best for the shareholders by doing what's best for Google. But that is an irrelevant argument, since there's really only three people whose opinions on the subject matter. then why say it.

If Google ever did do something along the lines of what you're proposing, they'd have to put a lot of time & effort into setting up a system that can't be easily abused by link spammers, is easy to use for idiots, etc etc etc. not really, if they already have a system to detect anyway. and it doesnt have to be that easy to use for idiots.

That may be more trouble than it is worth, compared to saying "not our problem, deal with it yourself." on a small scale maybe, but a search engine would want to return the most relevant results. helping sites get relisted would be good for google.

Talkorigins hacked by porn spammers

[Mouth of Sauron]

The site www.talkorigin.org is not the only site to have been de-indexed by Google.

This is a google cache of talkorgins.org showing the porn spam links.

However, I checked on deepx.com and it is *not* a porn site.

From DeepX.com's about page:

XML provides an open and flexible language for the creation, management and exchange of electronic content. Founded in 2000, deepX has an experienced team of consultants and developers, who specialise in the design and development of solutions using XML and the emerging technologies related to XML.

Also, another link shows www.theoi.com and it is *not* a porn site, either:

Here's how THEOI used to look via the Wayback machine.

Theoi.com has been banned by Google (no reason given) and forced to close down as a result. There are no plans to re-establish this site in the future.

wu.edu.gh is Valley View University is a Seventh Day Adventist college in Ghana.

Both deepx.com and wu.edu.gh redirect to porn sites.

Unsurprisingly, wu.edu.gh, theoi.com and deepx.com have been de-indexed by google.

I speculate that all these sites that have been de-indexed were tagged by automated processes.

No Free Consulting

[rossz]

Basically, this "so called" webmaster wanted free consulting from Google. I don't think so. My personal response would have been, "I'll be happy to supply you with the information you request. It will, however, cost you my standard consulting rate of $xx/hour, two hour minimum."

Only friends and family get free computer help from me, but I'm rethinking that policy since I spent half a day cleaning the malware off my brother's computer during the last family holiday. He probably won't ask me to do it again, though. When he asked how his system got so infected, I answered (in front of the entire family), "You got infected from all those lesbian porn sites you've been visiting."

Re:ahhh i love it

[denoir]

Why should it be the responsibility of ID to explain who created the creator?

Because it otherwise fails to explain anything. If irreducibly complex things require a designer then the designer who designed them will be even more complex. Since the designer theory can't tell us, well, anything, the only way to investigate is to go up the ladder: who designed the designer?

If you say that that's a metaphysical question that cannot be answered, why not just skip the whole designer/creator bit and say that you are not interested in physical modeling of the world. Invoking an extremely improbable super-being to explain the world is very unhelpful. That's what earlier civilizations did: thunder was Thor riding in his carriage in the sky etc

What the ID followers want is a return to that using the logic "I don't understand it so it must be God's work."

Re:Backups?

[grimJester]

That presupposes the site was intelligently designed. Starting with that kind of assumptions is completely unscientific.

Re:Words are Meaningless - Public Utility

[nacturation]

What you're missing is that Google gave him no clue/hint/guide/comment/help on why he was delisted. login: root
password: ******

Incorrect login for user "root". You got the first and fourth characters correct, and one other character was correct but in the wrong place. Please try again and/or make use of one of the following clues/hints.

You can also try one of the following non-root accounts:
1. admin (8 character password)
2. backup (6 character password, all lowercase letters)
3. johndoe (5 character password)
4. maryjane (7 character password)

Failing that, if you can't remember any passwords this server is located at 1234 Main Street, Anywhere, USA. The server rack key is located in the desk drawer on the second floor in the manager's office. You can boot with a Knoppix CD (inside the rack) and reset the password after mounting the hard drive.

Often, helpfulness is at odds with security.

Ethics: The users are our customers

[MDMurphy]

Google has been up front with where their loyalties lie in the search engine business: With the user. They got big and continue to be big because the give results that the search users are looking for. In general, this means the links they present are on the topic queried for and on the basis of links from other sites the content has been "rated" useful.

If a site is designed ( or screwed up ) such that it shows as a result to a query when inappropriate, delivers spam, or ranks higher than the content would warrant, and Google still presents it as a search result, then Google has failed their customer.

Webmasters are not their customers, individuals who are searching are. Ethics says that you give your customers what you promised them. Ethics says you live up to what your stockholders expect by doing what you told them you do: Delivering search results that keep your customers coming back ( and serving them up ads each time ).

Re:Words are Meaningless - Public Utility

[HUADPE]

PG&E is a public company. ComEd is a public company. Verizon is a public company. AT&T is a public company. They're all public utilities. Simply being a publicly traded for profit corporation doesn't mean that you're not a public utility.

These companies were all given special monopoly privileges by the force of government. They can run wires, pipes, and other items through your property without your consent, by law. They are required to provide service to all persons in their scope of operation by law. No such law exists regarding Google Inc. and they are not a utility.

Well I assume

[goldcd]

the revenue brought into google by having his site indexed, was considerably lower than the cost of having a google employee investigating his problem and givin him a site-audit.
Weird, eh?

Re:Words are Meaningless - Public Utility

[Columcille]

Going on the market puts some control of the company into the hands of shareholders, not the general public. Become a shareholder, then you can have a say and ask for a nice, friendly email.

Meanings are meaningless around here...

[WoodstockJeff]

It's funny how the Google apologists are always around on Slashdot to defend Google's (a private company) right to screw anyone, ...

It's also funny how the Google haters are also here to throw stones at every little perceived problem with how Google works. It's funny how they also seem to use Google a lot, despite of them. I wonder why that is. Could it be that Google does what they want it to do? And, is it Google's problem that so many sites have come to the conclusion that their very existence is tied to their Google page rank? If you do not like how Google works, don't use them, and use your site's robots.txt file to exclude them from indexing your site. The more people who use other search engines, the less "power" Google (or any other search engine) has over "the market".

In this particular case, Google gave the webmaster sufficient information to discover the problem. If it wasn't enough for "other honest webmasters", then they aren't particularly competent, in my opinion, which would tend to affect how I felt about their information being relevant, too. A lot of people spend a lot of effort trying to scam their way to the top of the page ranks. And it looks like Google is spending a lot of effort to keep the game "honest".

Google has no stake in my using their service, other than wanting to display advertising to me, just like a TV or radio station. Given that the website in question here is not a paid advertiser on Google, I don't see where they have a responsibility to do anything special for them. Their responsibility is to make money for their stockholders, the same as any other corporation. Their "niche" for doing this is to sell advertising that is displayed to people who willingly come to their site. Their way of making people come to their site willingly is to index pages in as "honest" a way as they can figure out to do. Refusing to index a particular site for dishonest links, whether intentional by the owner or not, makes them more desirable to most of their users.

And a few dozen people bitching about it in a front page story on Slashdot doesn't hurt, either.

Re: Some countries consider it hate speech.

[Black Parrot]

> There are a lot of hypocrites on that site. They claim that religious people are closed minded while completely ignoring anything the other side presents out of hand.

Can you call our attention to any creationist claims that have ever been made on talk.origins that didn't deserve to be dismissed out of hand?

> This blind faith in popular theories is not just restricted to theoretical physics but also appears in the biological sciences as well. Science is supposed to be a tool for discovery. It is not supposed to supply the meaning of life

Biology is no more concerned with the meaning of life than geology or meteorology is.

It's just that some peoples' world views are threatened by the facts that biology has uncovered.

> or delve into things which are best left to philosophers and theologians given our current state of technology.

I don't know of any question best left to philosophers and theologians. If it's not supported by evidence, it's just someone's opinion.

Welcome to the Real World

[Evets]

Have a peek over at the forums at WebmasterWorld, DigitalPoint, SearchEngineWatch, or any number of other webmaster related sites. This happens all the time. It is an issue that webmasters have had to deal with for some time now. Google at least provides some input for you if you can be bothered to register a sitemap with them.

Google has several billion pages in their index, and a significant portion of them are spam. Their business model relies on them having internal methods of dealing with web spam and it is not feasible or desirable for them to produce a list of violations to each and every person who runs afoul of their algorithms.

This is far from the most popular or important site this has happened to. Wordpress was delisted, as was BMW, Syndic8, and many others. This guy is using the controversial nature of his subject matter in an attempt to draw more attention. Get in line buddy, there is a long list of people whining all over the web about the same thing. Are you more important because the word Christianity is loosely affiliated with your site? Nope.

Do a little googling yourself and you can pretty easily figure out how to resolve the problem. It takes some time, and there are ways to accelerate the process. If you are that reliant on Google, it is time to start participating in some webmaster communities and figure out how to play ball with the Search Engines. Just like everybody else.

Google emailed this site

[GoogleGuy]

If you dig deeper, it turns out that Google emailed talkorigins.org to alert the site that it had been hacked and was stuffed with rape and animal porn spam. Google's head of webspam has posted a full write-up.

Re:Words are Meaningless - Public Utility

[edumacator]

they dont have to tell anyone how they found the problem, just where. if the webmaster of a site is deliberately trying to cheat google, they already know what pages are in offence anyway.

But if they tell the webmaster, who might be cheating, (remember, a lot of the exploits out there are actually used by the webmaster) where the problem is, then the cheating webmaster only has to get rid of one exploit and gains insight into the detection methods employed by Google. Then he can leave all the others in place. Wouldn't it be fair to say that the people doing evil is, well, the exploitive webmasters?

Don't hit reply yet...I know this guy was honest, but how in the hell could Google possibly tell who is legit and who is not? Google can't hope to be "fair," only just.

Re:Words are Meaningless - Public Utility

[Gulthek]

It would be the same as Microsoft stopping an application from running under windows.

Which would be an entirely appropriate response if said application was a virus.