Google DNS Glitch Caused Outage

An anonymous reader writes "Google suffered a pretty long outage saturday evening, due to some DNS glitches, according to company spokesperson. All Google services were down for a while, including Gmail and Google AdSense. There seems to be a DNS hijack, as some screen grabs show that Google.com was redirecting to another site, SoGoSearch.com. "

Comments

Whois Entries Not Indicative of a Hack

[LogicX]

Everyone keeps freaking out because when they run a whois query they get this:

GOOGLE.COM.SUCKS.FIND.CRACKZ.WITH.SEARCH.GULLI.C OM
GOOGLE.COM.HAS.LESS.FREE.PORN.IN.ITS.SEARCH.ENGI NE .THAN.SECZY.COM
GOOGLE.COM

This is NOT at ALL indicative of a hack.

All this means is that gulli.com chose to register a DNS server with their registrar called 'GOOGLE.COM.SUCKS.FIND.CRACKZ.WITH.SEARCH.GULLI.CO M' instead of ns1.gulli.com -- to do EXACTLY what they just did -- got your attention.

Simmer down everyone. If you whois ANY major site you'll see similar things. (Just try Microsoft.com)

Re:Whois Entries Not Indicative of a Hack

[A beautiful mind]

Also the Screenshots are just about BROWSER GUESSES. The screenshots show http://www.google.com.net!

You know, it's what happens when the browser can't find the given domain name (dns servers are down), that it tries www.google.com.com, then www.google.com.net and it happened to be already taken by the site in the screenshots.

Re:Whois Entries Not Indicative of a Hack

[Megane]

Wow, I thought that trick stopped working like four years or so ago. I even had one of those kind of entries, but took it out when the search stopped showing them.

Looks like these clowns aren't just limiting themselves to Google...

AOL.COM.IS.N0T.AS.1337.AS.GULLI.COM
AOL.COM.IS.0WNED.BY.SUB7.NET
AOL.COM.CANDICE-CHAMBERLAIN.COM
AOL.COM.AINT.GOT.AS.MUCH.FREE.PORN.AS.SECZ.COM
AOL.COM

Re:Whois Entries Not Indicative of a Hack

[AndroidCat]

Because, by default, whois does a search match on the entire record rather than just the name. Since the names of a domain's DNS servers are part of that record, some smartasses with spare domains load up the DNS server names with useless extra strings that will match lookups against popular domains like google.

This only confuses humans, and has nothing to do with Google's outage and overly helpful browser code.

Re:Whois Entries Not Indicative of a Hack

[AndroidCat]

There's a whois *program*? Damn, I've just been using telnet whois.internic.net 43 all these years!

Re:Whois Entries Not Indicative of a Hack

[Wieland]

Mozilla Suite: Edit -> Preferences -> Navigator -> Smart Browsing -> Domain Guessing

Firefox: Go to about:config and set user_pref("browser.fixup.alternate.enabled", false);

Re:Whois Entries Not Indicative of a Hack

[thsths]

Yes, you can append a . to the name. http://www.google.com./ will only ever get you google, or nothing at all.

This "trick" is a lot older than mozilla, it applies to all DNS lookups. It also prevents the name from matching a machine on the local network. Mozilla also seems to recognise the dot, and it avoids the "guessing" step.

Laugh!

[stabChmo]

So go search Google!

Google Web Accelerator

[Message Board]

Last night, Google Web Accelerator was accelerating just fine... except for the fact that when I tried to make it proxy google.com it told me that the web site wasn't available, and to try search Google for the site. Needless to say, that didn't work either.

Slashdot and Google

[brokencomputer]

Yeah and Slashdot was down with a 503 error yesterday for quite a while. But seriously, Google shouldn't allow this to happen.

Re:Slashdot and Google

[Chess_the_cat]

They don't owe you anything.

I wonder if Google's shareholders feel the same way or if they understand that they do owe their customers? They're a business; they owe me whatever it is I feel like asking for or I'll go elsewhere.

Re:Slashdot and Google

[jdgeorge]

I wonder if Google's shareholders feel the same way or if they understand that they do owe their customers? They're a business; they owe me whatever it is I feel like asking for or I'll go elsewhere.

Are you an advertiser on Google? If not, it sounds as if you are confusing what Google owes shareholders (return on investment) and their customers (advertisers) with what Google owes the user, (technically, nothing).

It is true that Google tries to provide a good experience for users, and that helps provide value to the advertisers and return on investment the shareholders are owed.

If, on the other hand, you are an advertiser, you should realize that Google's first obligation is to its shareholders, not its customers or its users.

(Okay, I realize that Google has other customers than advertisers, e.g. those who purchase Google's search services, users of Google Answers, etc., but my impression is that advertising generates the bulk of Google's revenue.)

SoGoSearch

[Dachannien]

I think it's far more likely that there are quite a few people out there with some sort of malware redirecting their failed DNS lookups to this site, as opposed to Google's DNS entry being hacked.

Re:SoGoSearch

[Dachannien]

Never mind. See posts below indicating that these schmucks registered the .com.net domain and have a host named "google" in that domain, hence google.com.net.

Re:SoGoSearch

[AndroidCat]

Google's DNS was down, browsers did something that most people don't expect. Nothing to see, move along.

Pre-FP

[LogicX]

Ironically people have been freaking out about this, even before slashdot posted the story; leaving comments in other articles

SoGoSearch didn't hijack

[Anonymous Coward]

SoGoSearch didn't hijack Google's DNS. They registered a domain name google.com.net. Because the browser couldn't find google.com it tried as google.com.net. It has nothing to do with them hijacking any DNS.

I do think it is unethical to register a domain such as google.com.net if you are not Google, but that is a different thing.

Re:SoGoSearch didn't hijack

[ryanjensen]

Thing is, they didn't register "google.com.net" - they registered "com.net". The "google" part is called a wildcard, and any "*.com.net" would go to SoGoSearch. (See this report about yahoo.sex.com).

The real problem lies in web browsers that append ".net" to a domain name when the .com version cannot be accessed.

Re:SoGoSearch didn't hijack

[Gollum]

In fact, I think they registered com.net, and simply created a wildcard DNS result for anything under that, which points to their search page.

As the parent says, it is common behaviour for browsers to try appending common TLD's to the end of an URL that is not found verbatim. When Google went away, the browser appended .net to google.com, and ended up at *.com.net.

A bug that people seem to be ignoring is that whatever browser is shown in the screenshot did not show the correct URL after the .net was appended, but left the original URL in the location bar.

Re:SoGoSearch didn't hijack

[Kristoffer Lunden]

A bug that people seem to be ignoring is that whatever browser is shown in the screenshot did not show the correct URL after the .net was appended, but left the original URL in the location bar.

Looks like Safari. And you're right, that's the real problem here, the redirect should be shown at the very least by changing the URL in the location bar.

Re:SoGoSearch didn't hijack

[jmaslak]

Uh, no, this is not a browser bug.

Yes, IE does do some bizare stuff on its own, but this is a RESOLVER issue.

Let's say you have a domain called "example.com". Let's say you have a host called "foo.example.com". What happens, with the common configuration, when you telnet to "foo.example.com" from a machine called "bar.example.com"? Well, if your resolve.conf contains search example.com, it will try to look up foo.example.com, then foo.example.com.example.com, then foo.example.com.com. The relevant section from resolv.conf (5) on my RH9 box:

Most resolver queries will be attempted using each component of the search path in turn until a match is found.

IE has a different broken component, but that doesn't come into effect until AFTER the resolver does its thing - it appends a bunch of TLDs to the name, not just ones in your search path. But IE does show you the proper URL when *it* (instead of the resolver) does this.

Yes, this all is very dangerous behavior and some systems have learned "Only append the primary domain, not each subdomain". I think Windows is one of these systems actually. That's why if you are quux.baz.example.com, a Window's machine can't telnet to foo.example.com by simply typing in "telnet foo". This was a departure from the Unix resolver rules for security reasons - a good idea IMHO. (if a user typed "secureexampleintrant", you wouldn't want a phisher out there to have set up secureexampleintrant.com so your user unknowingly goes there instead of secureexampleintranet.example.com). Of course I am of the opinion that these "DNS shortcuts" are bad in general and the search kewyord in BIND needs to go away. Let people type the whole URL.

A little DNS knowledge is a dangerous thing indeed, though, as there is all sorts of FUD with this. This is DEFINATELY not a browser problem.

Re:SoGoSearch didn't hijack

[Dun Malg]

com.net, net.com, etc should be reserved.

A better idea is to not have such brain-dead DWIM "features" in the browser. What kind of stupidity is it to blindly append a TLD to a URL that already ends in a valid TLD?

Re:SoGoSearch didn't hijack

[autocracy]

It's definitely a browser problem. The resolver doesn't do that... the browser makes the other requests after being told NXDOMAIN by the resolver. So, while the issue comes from getting the wrong DNS response, it's because the browser asked the wrong questions thereafter. This also doesn't have to do with search directives. I'm sure there's something you're saying that I'm calling differently than you mean, but it's still an issue of the browser in this case.

Re:SoGoSearch didn't hijack

[Karma Farmer]

No. You're confusing the "resolver library" code used by the program, and "resolving name servers" network services. He's describing the built in search feature that many resolver libraries use, you're describing the part of the network protocol that library uses to communicate with the network service.

It's worth asking if Mozilla and Firefox use the "default" resolver of the host operating system, or if the developers took the "path of greatest suprise" by including one of their own.

Re:SoGoSearch didn't hijack

[jmaslak]

RTFM!

This has been default resolver behavior on Unix (including Mac OS X and Linux, IIRC) since early versions of the resolver libraries.

I am NOT talking about the DNS server itself, rather the client libraries.

On a Linux machine (at least RH9), look at
"man 3 resolver".

Note the "RES_DNSRCH" option:

"If set, res_search() will search for host names in the current domain and in parent domains. This option is used by gethostbyname(3). [Enabled by default]."

Note also that it is enabled unless someone turns it off in the code of the calling application.

Note that "gethostbyname" is the common way in Unix C programming to find out the IP address of a DNS name. And gethostbyname() *is* using this option.

If someone wants to disagree with this, I'm going to say "read the source" and then post that source to disprove this.

In this case, when google.com returned NXDOMAIN, the resolver (accessed by gethostbyname on OS X and Linux) then looks up google.com.localdomain.tld. It then looks up google.com.tld. So, if your local domain was "example.net", it would try google.com.example.net followed by google.com.net - exactly the behavior we saw yesterday. It also explains why some people got "page not found" instead (there is no "google.com.com" - no DNS wildcard under com.com)

Not a hijack

[Kip]

They were just taking advantage of browser behavior.

www.google.com.net leads to sogosearch.com

When a browser fails to resolve an address, they will try adding .net and .com to the end of the address on the assumption maybe the user forgot to add it.

Re:Not a hijack

[omb]

Yes, and every time we add 'tard' support to our
code we add another potential _exploit_.

Has it gotten to this point yet?

[fwice]

Are people really this dependant on google that when there is an outage, people really flip out?

I mean, there are other search engines.
Other email services.
Other mapping things.

Seriously, what were people doing a couple years ago? If your life is that in tuned to google, maybe its time to 'log off' (and pardon the cliche).

Re:Has it gotten to this point yet?

[Anonymous Coward]

Yeah I'm bad for that. During the outage I tried to Google for another search engine but...

Re:Has it gotten to this point yet?

[Poeir]

Google offers search results via SMS (text message 46645 with your query). It also has Google local, which means you can search for telephone numbers. I don't know of another search service with this functionality, and I attempted an out of state lookup during the outage without knowing about it. I actually did get results much later, but they weren't useful then.

A couple of years ago, I wouldn't have looked up the number at all, but I also wouldn't have been used to being able to look it up at any time.

Re:Has it gotten to this point yet?

[telstar]

"What's stopping you from using another web-based e-mail account, or using your ISP's e-mail service?

You mean, other than that not solving any problem? If the email service you use goes down, and you don't retain a local copy of that email, you immediately lose access to a wealth of information. Doesn't matter if it's GMail, Yahoo!, Hotmail, or whatever. I don't see how your suggestion solves the problem.

Re:Has it gotten to this point yet?

[jmaslak]

I'm sorry, but "important" email being sent to a free email account?

If you get important email, I suggest paying for an account that provides support as part of the price. "Free" doesn't typically mean "great support", not even in the case of Google.

Just a DNS glitch

[Eric(b0mb)Dennis]

Lots of rumor of DNS getting poison and/or google site getting hacked. The reason benig is people thought google.com was going to SoGoSearch.com..

But apparently it was just their browser's not finding google.com and trying to go to Google.com.net

Stop flipping out!

Just noticed

[vivekg]

Thought gmail was slow and Adsens was not working but google.co.in was up and running :)

However I noticed http://www.google.com/intl/xx-hacker/ don't know what the hell it is... or just one of those google own funny stuff :-?

So the DNS was down...

[Karakth]

Just 216.239.57.99 it.

Re:So the DNS was down...

[srblackbird]

I have 2 DNS bookmarks for Google.com, and other website I visit frequently
In case there is an attack at the DNS-servers.

http://216.239.39.99/ and http://216.239.57.104/

Re:It's time to end our dependence on google

[EllF]

...or perhaps we reject Microsoft because we disagree with its corporate goals, and find its products to be substandard, while agreeing with Google's, and find its offering to be exactly what we want?

Re:It's time to end our dependence on google

[mattkinabrewmindspri]

Because Google doesn't suck. There really isn't anything that seems to compare.

And if there is, please, show us. I'm interested.

Monopolies aren't inherently evil. Monopolies that use their position to hurt consumers are evil, but I don't know of Google doing that.

Google didn't cash 400,000 US$ during that time

[astrab]

According to gigaom.com, Google acknowledges having suffered a 'DNS blackout' for two hours (aprox) this past Saturday, and users couldn't access the search engine.

During Q1 2005, Google cashed $657 million by showing sponsored links on search results. This means 300,000 US$ per hour. Taking into account that this issue happened on Saturday (less users), we can estimate the 'non-revenue' figure in 400,000 US$ aprox, without considering other non-working services like Google AdSense, which probably suffered problems during this time.

http://google-blog.dirson.com/post.new/0260/

Re:It's time to end our dependence on google

[dfjghsk]

Google with it's 85% market share. Google with its total control of the web search market.

Except, its market share is only 35%.. which is far from a monopoly. (For comparison, yahoo is at 32%)

Only here on slashdot does everyone think google completely controls the web search market.

This hit Microsoft as well

[Nichotin]

Didn't anyone notice?

Re:This hit Microsoft as well

[Nichotin]

Seriously, it was not meant as a joke.

Those schmucks were first

[AndroidCat]

com.net: Record created on 28-Sep-1994

google.com: Created on..............: 1997-Sep-15.

Re:It's time to end our dependence on google

[91degrees]

Microsoft let me develop software for Windows for free. They even offer online help. Come to think of it, I'm sure they would have no ibjection to me giving them software for free as well.

Of course Google let you submit a site for free. Their whole business model depends on it.

For Microsoft...

[frostman]

I just tried Microsoft. Hilarious.

frost@louddrunk ~
$ whois microsoft.com|grep MICROSOFT
Server Name: MICROSOFT.COM.WAREZ.AT.TOPLIST.GULLI.COM
Server Name: MICROSOFT.COM.WANADOODOO.COM
Server Name: MICROSOFT.COM.SUX.BUT.PYROFREAK.ORG.RULEZ.AND.DIOX YTECH.NET.DELETED.GANDI.NET
Server Name: MICROSOFT.COM.SMELLS.SIMPLECODES.COM
Server Name: MICROSOFT.COM.SHOULD.GIVE.UP.BECAUSE.LINUXISGOD.CO M
Server Name: MICROSOFT.COM.RAWKZ.MUH.WERLD.MENTALFLOSS.CA
Server Name: MICROSOFT.COM.OHMYGODITBURNS.COM
Server Name: MICROSOFT.COM.LIVES.AT.SHAUNEWING.COM
Server Name: MICROSOFT.COM.IS.NOT.AS.COOL.AS.SIMPLECODES.COM
Server Name: MICROSOFT.COM.IS.IN.BED.WITH.CURTYV.COM
Server Name: MICROSOFT.COM.IS.GOD.BECOUSE.UNIXSUCKS.COM
Server Name: MICROSOFT.COM.IS.A.STEAMING.HEAP.OF.FUCKING-BULLSH IT.NET
Server Name: MICROSOFT.COM.HAS.ITS.OWN.CRACKLAB.COM
Server Name: MICROSOFT.COM.HAS.A.PRESENT.COMING.FROM.HUGHESMISS ILES.COM
Server Name: MICROSOFT.COM.FLINGS.POO.AT.MONKEYCORE.COM
Server Name: MICROSOFT.COM.FILLS.ME.WITH.BELLIGERENCE.NET
Server Name: MICROSOFT.COM.CAN.GO.FUCK.ITSELF.AT.SECZY.COM
Server Name: MICROSOFT.COM.ARE.GODDAMN.PIGFUCKERS.NET.NS-NOT-IN -SERVICE.COM
Server Name: MICROSOFT.COM.AND.MINDSUCK.BOTH.SUCK.HUGE.ONES.AT. EXEGETE.NET
Domain Name: MICROSOFT.COM
Domain name: MICROSOFT.COM

With google down..

[kun]

With google down who's going to raise my children!?

The set of valid TLDs changes

[tepples]

What kind of stupidity is it to blindly append a TLD to a URL that already ends in a valid TLD?

When ".museum" was first added, how would existing browsers know that it is a valid TLD?

Just think how this affected ISP help desks

[IronChefMorimoto]

Imagine all the people who have Google.com set as their homepage when they start up a web browser. I can't imagine what happened to ISP help desk lines when Joe Bob Family Man hopped onto his computer Saturday night to check a golf score only to find a 404 error or some "page not found" error when he fired up MSIE.

Think about it -- Google just doesn't go down. Not like some websites. It's so simply designed, and in some people's minds, that means it can't fail.

Hell -- I stupidly went into my Linksys router interface after FireFox gave me a startup error to see if my ISP had dropped my connection. I didn't think to look at CNN.com or another website (which were working fine, so NOT an outage). Why?

Google just doesn't go down. Reliance is a real bitch sometimes, no?

IronChefMorimoto

Re:Good example of why SPF's security holes

[LogicX]

What does SPF have anything to do with this?

If your domain is high-jacked due to a fault with the security of your domain registrar, then yes, you have bigger problems than any anti-spam solution.

This is not the purpose of SPF

If you read spf.pobox.com You can learn that SPF is merely designed to be a system which can eliminate domains being spoofed in the from field of spam messages.

If someone is using one of my domains (logicx.net) to send spam; I can reduce the affect of such a joe-job attack by having a published SPF record; such that receiving systems can verify if the email came from a logicx.net mail server, and reject it appropriately.

SPF and PGP have entirely different authentication approaches. I'd go so far as to say that PGP is more integrity checking.

SPF is a verification that mail for a particular domain came from an appropriate server -- with the goal of disposing false emails (spam, spoofs, etc.)
This is not at all a system to verify users on that particular email system.
This is where PGP steps in -- It is used to verify the integrity of the email -- that it came from a particular user, and came unaltered.

Finally, where has it been verified that their was a breach of their DNS system?

All of the screenshots have now been confirmed to be a firefox situation where when DNS failed it resolved www.google.com.net -- which resolved to the people who own com.net

none of the regional googles

[DJCF]

Actully, none of the regional googles were affected -- google.co.uk, google.co.th, etc.

I for one,

[Aeron65432]

Welcome our Google-slaying overlords.

Google DNS Glitch Caused Outage

[caluml]

Google DNS Glitch Caused Outage

I knew that . Where is the full detailed breakdown?