Slashdot Mirror
How Many Google Machines, Really?
BoneThugND writes "I found this article on TNL.NET. It takes information from the S-1 Filing to reverse engineer how many machines Google has (hint: a lot more than 10,000).
'According to calculations by the IEE, in a paper about the Google cluster, a rack with 88 dual-CPU machines used to cost about $278,000. If you divide the $250 million figure from the S-1 filing by $278,000, you end up with a bit over 899 racks. Assuming that each rack holds 88 machines, you end up with 79,000 machines.'" An anonymous source claims
over 100,000.
No wonder I'm'a Googlin'
Here's what I do: Bitty Browser & Andromeda
* of servers in the world
* of servers in the USA
* of servers running Linux
That's $3159 per machine, and those are today's prices... They weren't so low a couple of years ago...
Can you imagine a beowul.... oh.. wait..
SysWear - Geek T-shirts (UK/Europe)
1) google is so pretty and smart
2) google is worth so much money
3) google has a huge rack!!
There was an article recently about how Google constantly understates various statistics about itself to mislead potential competitors. This article also said that the SEC would not allow them to do this once they became a publically traded company.
From the Article: "Google is managing between 45,000 and 80,000 servers"
Last time I checked, there was a pretty big difference between 45,000 and 80,000 servers. I mean, 80,000 is almost twice 45,000.
Cheers,
--
tom
- tom -
Seriously? What is the point of this article? What next? Linus found to prefer blue ink, over black ink?
========
CINC, 4th Penguin Legion
For all the econ majors out there. What exactly does Google have to tell us now that it's an IPO?
Ok, its not an interesting question, Im just hoping for first post.
------
"Will the highways on the Internet become more few?" --George W. Bush, in Jan. 2000
I don't think this is that strange: after all, that 10,000 machines figure is several years old. It's only logical that Google has expanded their facilities since then.
This space intentionally left blank.
SCO now knows how big an invoice to send Google! :-D
My rights don't need management.
I hang around too many old-timer mainframe geeks. MVS forever!!! and such.
I'd be more interested in the specs of those machines rather than the number of machines. Wonder if that will ever be revealed. :)
Downmix - The Artscene News Source!
Of course, someone *will* say the obligatory "imagine a beowulf..." comment in this thread.
Think of how many racks filled with nothing but data storage they must have. This only counts the actual dual-processor machines, so there is bound to be tons of computer stuff that isn't even counted.
According to calculations by the IEE, in a paper about the Google cluster, a rack with 88 dual-CPU machines used to cost about $278,000
Um, don't you think if you were buying 899 racks you might actually, you know, negotiate for a better price?
This isn't the only assumption in your analysis, and the problems with them will be compounded. What's the point of this, really?
I was always under the impression that Google used a lot of "cheap" hardware. Meaning, they only used IDE and non-rackmount machines.
So, they probably don't used "racks" but if they were, that means they could only get about 12-15 desktop machines (single proc) per rack. That's a whole lot less than 42 - 1U rackmounts to fill the rack.
Might just be me, but damn, don't you think this has raised the interested of our three letter entities? i mean, damn that is just some serious computing and indexing power on cheap, "disposable" hardware...with a filesystem that can keep track of that many machines? If i headed one of such entities, i'd sure want to know more about it!
My guess is just as your guess which would be:
your guess + 1 = my guess.
We already know they have enough servers to saturate a T1000 line so might as well stop here and talk about something more constructive.
Remember there's a little thing called "volume discount"...
It's gotta be more than that.
There are two kinds of people in the world: Those with good memory.
That can add 40% to the cost of a cluster.
With all those TFlops, no wonder Google converts units so quickly.
... it's full of stars"
You mean the PigeonRank(tm) technology is a hoax?
Does the SEC filing mention whether theses are datacenter boxes or employee workstations? Beyond that, do they take into account storage array costs, telco equipment, etc? Not that I don't love speculating about the awe that is the google datacenter, but an SEC filing seems like a fairly inaccurate, roundabout way to make an assumption about cluster nodes . . .
With that much computer power at their disposal they could do some cool things - maybe some sort of distributive computing thingie or big database of some kind. Of course they would have to network all those systems togther - that could be expensive. Imagine how much a 80,000 seat license for lantastic would cost!
Humor from a Genetically Molested Mind
because with ~80,000 machines, they can easily put a few hard drives in each, and give everyone 1gb of gmail space... I didn't think it was possible.
where do you go to buy 80,000 hard drives?
Runnin' On Empty
In your standard 42U cabinet, you're talking a half-U per server. Umm.. not happening. Let's just say I happen to know they use 2U servers, for a total of 21 per cabinet. Custom jobs - just the "floor pan" (i.e. no sides, or top for the case), system board, power supply, and I think a single (or possibly dual) hard drive (I didn't want to be too nosy staring into someone else's colo space). Oh, and network. And rumor has it, they're putting in close to 200 cabinets in just this location alone.
How abt a file search on google? Does it will bring dooms day for software makers?
This many computers must use quite a bit of power and they probbably also need some serious airconditioning. I sure wouldn't want to receive their electricity bill by mistake. :)
I wonder if google will start up a web-hosting business? I bet you can't beat their uptime guarantees. They could provide sql, cgi, etc, and build in multi-machine redundancy for your data just like they do for theirs. It'll be the google server platform, just one more step to replacing Microsoft as the evil monopoly.
Surely there must be some Google tech out there reading this.. Just post AC and tell us.
The number of machines Google uses is considered a trade secret. By attempting to determine how many machines they have, you're in violation of the DMCA. I'm calling the FBI.
ows SCO over $90million USD assuming that all of those machines run Linux, which they probably do.
100% of the money isn't devoted towards servers and rackspace. There's operational costs, network infrastructure, bandwidth, software costs, maintenance contracts, spare parts... it ain't all just hardware.
working at abovenet google has pulled there machines in and out of our data centers many a times. its incredible the way they have there shit is setup.
they fit about 100 or so 1u's on each side of the rack, there double sided cabinets that look like refrigerators. there seperated in the center by noname brand switches and they have castor wheels on the bottoms of them. google can at the drop of a dime roll there machines out of a datacenter onto there 16 wheeler, move, unload and plug into a new data center in less than a days time.
Since the 10k server number was first floated, I believe google has added quite a few, meaning 6 to 10 whole new datacenters around the world.
It would only make sense that the server count would now be in the ballpart of what is mentioned here.
Google hasn't been standing still, and I've heard the "Google has 10k servers" for 1-2 years now.
-Pete
Soccer Goal Plans
Does it include additional costs like maintenance costs, administrator salary, electricity, etc.
At an estimated 300W per machine and 7 cents a KWH, that's $14M just for the electricity to run those machines, not including air conditioning for that many machines. So is the figure quoted JUST for the computing hardware?
>where do you go to buy 80,000 hard drives?
You call IBM.
I heard they have a large amount of 75GXP drives ready to rock'n'roll[0]!
(fineprint)[0] Raid-1 capable interface not included.
a cluster costs more then just machines
Couldn't we just ask google?
Cheers,
RoadkillBunny
...assuming 200W per server, which is probably low, but probably compensates for 79,000 being most likely an overestimate. However, that doesn't even begin to account for the energy used to keep the stuff cool.
Anyone know how many trees per second that would be? Conversion to clubbed-baby-seals-per-sec optional.
Please help metamoderate.
A Pentium 4 dissipates around 85 W of heat. I don't know what the Xeon does, but let's be kind and say 50 W (wild guess). Using the article's "low end" estimate, that brings us to 4.7 MW!
I hope they have good ventilation...
Since it is known that Google has the largest installed base of Linux and now they are about to go IPO in the billions, I wonder why SCO has not gone after them? Apparently, it is not use of Linux that makes SCO persue a company.
The interesting thing is, that if SCO really has MS backing and MS is pulling strings, then I would think that MS would want SCO to persue google to tie them up for awhile.
I prefer the "u" in honour as it seems to be missing these days.
42.
How are they fitting 88 dual cpu computers in a rack? Last I checked, racks were between 42U and 46U. That is 42-46 1U computers. I have seen 1/2 U computers, but never dual cpu 1/2U computers. I can see 88 processors per rack, but not 172 processors per rack. Plus, when you have such a high density of computers, you need lots of power and cooling. Most racks have UPSes and cooling fans that take up rack space.
In every other Google article I've read, they say that they use old slow PCs in mini to mid tower cases. You can fit about... 12-16 machines in a rack if they're not rackmount cases. Not 88.
In any case, all this calculation is a bit over speculatory and pointless.
I just wasted your mod points! HA!
I was going to start my own Google clone, and after I figured out how to crawl the web, index it, and cache it, and how to do the keyword-based advertising, and after I wrote the programs for it, which were actually pretty standard things out of the academic literature, the only thing left I needed to know was the exact number of machines they have. Now I have a much smaller search-space for that number. Bwa-ha-ha!
If the statistics are lumpy when they reach the SEC, they will be lumpy when they reach you.
n/t
When you can just open "Computer Architecture: A Quantitavie Approach, 3rd Edition" by Hennessy and Patterson to page 855 and find out that in summary: ...
Google has 3 sites (two west coast, one east)
Each site connected with 1 OC48
Each OC48 hooks up to 2 Foundry BigIron 8000
80 Pc's per rack * 40 racks(at an example site)
= 3200 PC's.
A google site is not a homogenous set of PC's instead there are different types of PC's that are being upgraded on different cycles based on the price/performance ratio.
If you want more info get the patterson hennessy book that I mentioned. Not the other version they sell. This one rocks way harder. You get to learn fun things like Tomosulo's algorithm.
If I am violating any copy rights feel free to remove this post.
The reason this is considered funny (perhaps not by moderators, but in general) is why the Swedes took to the streets for the abolishment of IP harassment yesterday.
What are you doing?
Interesting People 2004/05:
I know for a FACT they passed 100,000 last November. One thing the Louis calculation may have missed is Google's obsession with low cost. For example read the company's technical white paper on the Google file system. It was designed so that Google could purchase the cheapest disks possible, expecting them to have a high failure rate. What happens when you factor cost obsession into his equation?
first off, don't they list the 10,000 figure as number of machines per data center? and i know they have at least 5 or 6 data centers around the country. so as far as i'm concerned, these numbers still add up...
in how they recycle their gigantic heat output...perhaps move data center to the windy city, open up a homeless shelter next door, and put the hot air to good use for once. They might even get a tax break on this.
Better yet, open up a nursery (plant type) next door , build a green house, and piple 25% of the heat to it. Have you guys see the price of trees lately? Google could make a killing with the "recycling" plant.
All those machine, all that complexity and activity, all boiled down to one little box under a Google logo. The most useful input box on the internet.
Thanks Google!
Can we blame them for the rolling blackouts? ;)
...due to simple arithmetic: if a std 42U rack was full of 1U, dual proc systems (42) that's 84, not 88 cpus AND CERTAINLY not 88 DPs (196 cpus).
Now normally you don't fill all 42U with systems either -for thermal and other logistic reasons.
Probably, your 899 racks (assuming you're right about THAT) would really hold about 35 systems so you're at about 31465 systems (62930 cpus).
- Keep costs down; and
- What happens inside the company, stays inside the company.
Figuring out the number of servers they have is why we're noodling over the second point, but the first point is what probably as us all thrown off. Someone in a position to know said recently that he could state as a an absolute fact they have more than 100,000 servers -- and added that merely mentioning it probably violated multiple NDAs he had."It was a summer's tale: Just a boy, his Linux, and a head full of dreams..."
The CIO and Head Brainsurgeon (he really is a medical doctor) was at SVLUG last year he said there were about 11500 Linux boxes at Google.
I knew a woman who "reverse engineered" the lines on the palm of my hand and determined that I had 10 inches of manhood.
I think she went to the same school as Slashdot's engineer.
<bart
1. Implement a Beowulf Cluster
2. Index Soviet Russia before they index you
3. Index images of panties and Natalie Portman
4. Profit!
Table-ized A.I.
This is more of a technical question than a business one, but what exactly is the "choke point" in the server side of a Google search? Sure, those 10,000+ machines have no problem obliterating even complex searches. But surely there must be "gateway" type machines which route the searches. How does Google prevent these from being overloaded? (Or is that a trade secret?)
They better have at least 10^100 machines, or they will be getting a call from my lawyers.
word.
Did anyone think of the electricity needed to power and cool 50,000 servers
. html
The 1,100 Apple cluster at Virginia tech uses 3 megawatts, sufficient to power 1,500 Virgina homes
http://www.research.vt.edu/resmag/2004resmag/HowX
Yes, it is true: every time you hit Google, you are polluting the Earth.
The next pasture is always greener
A major thing wrong with those price calculations is that fact that Google has probably neen buying a few (hundred) machines a month for the last few years. Also old machines (and some new ones) die and are retired. So the cost is probably very messy. Presumably when they back a bunch they go for the best price/performance ratio at the time. As the years have gone you get more cyles per buck.
Here's the Google Cache of that page. The link returns a 404 for some reason.
Any formula to calculate how many google stocks the slashdot owners bought ?
Now this figure seems pretty low to me. Even if you double it, considering one cluster on each coast, you're still not near what others have speculated.
Ah, how much do you think a machine room costs?
How much for UPS provision?
How much for cooling?
How much for LAN/switching?
While individual machine costs may well be low these days, the infrastructure to support them is *not* cheap.
I reckon the estimates of the numbers (40k, 79k etc) of machines based on that 250 million dollar figure *seriously* overestimates the numbers of actual computers.
Government of the people, by corporate executives, for corporate profits.
A good gamma radiation blast could turn those 100,000 machines against us! The once great internet oracle could turn and wipe out humanity.
"GOOGLe" I see the 3 6's.
Isn't it scary that according to these figures, Google's datacenter should theoretically be able to DDOS the entire Internet?
Someone mentioned that they have enough bandwidth/processing power to saturate a T1000 line. Scary...
-- If you try to fail and succeed, which have you done? - Uli's moose
So tell me, do you have a script that automatically makes useless mirrors? How come we never see mirrors of things that actually get brought down?
Douglas P. Price
Wouldn't it have been easier just to have put up a couple of racks running traditional Unix Big Iron, instead of wasting all that money on lots of crappy little Intel boxes?
i must have gotten dumber over the years, but i ...
... ..errr... another bubble?
...
honestly think google used to do a much better job
maybe two years or more ago
it's not how many computers you have but how fast
your backbone connection is, how "good" your
database is and how efficient your search/index
algorithem is
anyways methinks this "how-many-computer" thingy
is just to boost the initial IPO offering price
since just going public for some MELLIONS because
of some data on HDD that might just be obsolete in
a few weeks is
IF google goes public for MELLIONS you can count
me in on the search wars, since the amount
everyone is ready to pay for a google stock is
mighty redicilous!!!
maybe googles has "revers-kick-my-butt" problems
since they're putting their own ads on some pages?
dunno
The cost of acquiring the machine is a fraction of the cost of owning it.
And lets not forget the overhead of 2 networks per machine and all the patch panels, wiring, switches. Toss in console management (which may not be on all machines at all time), monitoring and management of said machines. Oh, and one really tired guy running around.
Disks are going to fail at a rate of several hundred or thousand PER DAY, just statistically. (along with power supplies etc)
Toss in that in three years, ALL of those machines are obsolete.
That's huge.
I've got ~300 racks in a half full data center upstairs from me. All network cables run to a room below it to patch panels. Around 50% the size of the DC is cable management. Next to that is a room FILLED with chest high batteries - these are used during outages until the generators need to be kicked on. And a NOC takes up about 1/5th the space of the DC (monitoring systems worldwide, but it's got seating for maybe 40 people - tight and usually filled with 10 folks, but in a crunch we live up there).
So that $3159 is only a bit of it. And in 3 years, all those machines will likely be replaced for whatever $3k buys then. That's about to be a 2 CPU Athlon64 box. If Sun can pull a rabbit out of its ass, we'll have 8 and 16CPU Athlon64 boxes. At least with that, some of the CPUs can talk to each other really really really fast.
I read an article on The Register saying they laughed at Intel when they wanted them to use the 200W Itanium chips. Power/heat ratio and low cost are the single biggest factors in Google's hardware decision.
Do the math. Gigablast.com said they use 8 machines (each P4 2.4, 2Gb ram, lots of hdd) and they handle an index of 250Mil pages.
So, of the search engine part of google has like 4.2 billion pages, that would scale up to 135 machines!
So with 200 machines they could do the job. OK, that is not completely correct because we didn't account for the difference in traffic.
Gigablast says it handles 50searches/sec on 8 machines - 6.25 per machine. Google has 200million searches/day that is approx 2400/sec so that would scale to 384 boxes. Still way lower that 45,000.
So I really don't get why they need so much power.
He displayed a little numerical dyslexia... it's 359 racks, not 539 for $100 Mil. which makes the stats a little different: 31592 machines 63184 CPU's 63184 GB RAM 2527.36 TB of Disk space and I'm not sure what his logic is behind the Teraflops calculations... looks like he's taking 1Ghz==1TFlop which would give about 126.4 TFlops. Aside from that error, the figures sound pretty realistic to me. But I wanna know how much bandwidth they use.
Sounds like a pretty stupid idea to me. Lego is expensive stuff.
were you expecting to see a sig here? perhaps you'd rather see the inside of an ambulance!
In your standard 42U cabinet, you're talking a half-U per server. Umm.. not happening.
One word: blades.
I'm not saying that's what Google uses (I have no idea), but that is how to cram servers in at less than 1U per server (eg 8 or more server blades in 4U of rack). There are also some 1U boxes that cram 2 servers in side by side.
-- Alastair
Google is building their trusted computing grid, where the brute power of cheap hosts are used in a global trusted matrix. It's there software that is amazing and builds this huge cluster around the world. There's no point in guessing how many machines they have now, the number rises every second. But sooner or later whether you want it or not, we will all become a part of one fancy trusted grid where we (hopefully) all share resources. If it will be Google's grid or my grid? Who know... just my 2 cents.
What did one Google machine say to another Google machine?
- Don't worry, I've got your backup.
Why did the Goggle machine cross the road?
- To get to the other cluster.
Geez dude, go back to school and learn how to punctuate properly and the proper use of there/they're/their. I'm not a grammer/spelling nazi. Even though mistakes annoy the shit out of me, I usually let it pass. I know I make the occasional mistake myself. But your post was just too much.
I don't know why I'm doing this, but here's a corrected version of your post:
Over 100,000 servers? Amazing... I wonder then how many pigeons they must have, and where do they get them all?
Given how fast Google is, we expect that they keep all the text of all the web pages that they index in memory. If we estimate 100K machines and 4,285,199,774 web pages, that's 42,852 web pages per machine. Let's guess 1 GB RAM per machine, then that's an allocation of about 25 KB per page (quite a bit larger than the average page size, I suspect). Of course, they've probably replicated the web a few times; let's guess 3 times, so that's about 8 KB per page -- still room to spare, and it's possible that the average memory per machine is greater than 1 GB. Plus, they could compress less popular pages -- the delay of decompression in memory is probably small.
Of course, once you consider that they keep thumbnails of al the images they index, things get tight very quickly. Plus, we can't forget the actual INDEX from words to documents -- that's in memory, too. And Orkut (which is probably pretty small, come to think of it).
GMail is another story altogether. 1 GB per user for 100K users would saturate their cluster. Plus indexes for searching mail. It seems unlikely that we'll have all-memory mail accounts anytime soon.
One of these days (in the next 10-15 years) my PDA will have as much computing power and storage capacity as Google does today.
Amazing magic tricks
Yea, That'd be a good idea, web hosting. Oh, and they could offer e-mail too. Wait..... I think they are working on that.
Google executives sir,mam,person, do you mind if you could lend me a few boxes?
(\_/)
(O.o) This is Bunny. Add Bunny to your signature
(> <) to help him achieve world domination.
I work at a major investment bank and I've had a chance to see how procurement on a large scale works. I've seen them get servers / disk / etc at 50% to 65% discount in volume. It seems that when you buy a 1000s of anything directly from the manufacturer, they will bend into pretzel shapes to sell it to you at a huge discount.
google is starting to resemble the wonka factory from willy wonka and the chocolate factory.
Here's what actually is happening...
:-)
Google is running on YOUR MACHINE, and everyone elses machine that is connected to the internet. Through very sophisticated software and anti-detection techniques, Google uses your 'spare' CPU cycles to search, index, and respond to queries.
Has anyone actually ever SEEN any of Google's machines?
Scary, huh?
"Your phone company can't just lose a few calls you made and not bill you for them."
Wait, what's wrong with that one?
G
Let's /. Google everyone!
Click Me!
That's right. All your base.
If you're not a competitor, why does anybody care how many servers Google uses? Is there a light bulb joke in here that I'm missing?
Yahoo!
His pricing in the summary may be a bit off.
:)
:) Well, not the Asus 1400r, those are built into a 1u case, but other machines we've built for servers are very easy to build into midtowers instead. Those machines don't get gobs of memory, but do get extras like nice sound cards and CD/DVD players. The price would be the same, as they'd probably still be attaching them to the same networking equipment. 132,000 servers, and 2,682 workstations and dev machines is probably fairly close to what they have.
Every article I've read about Google's servers says they use "commodity" parts, which means they buy pretty much the same stuff we buy. They also indicate that they use as much memory as possible, and don't use hard drives, or use the drives as little as possible. From my interview with Google, they asked quite a few questions about RAID0, RAID1 (and combinations of those), I'd believe they stick in two drives to ensure data doesn't get lost due to power outages.
We get good name brand parts wholesale, which I'd expect is what they do too. So, assuming 1u Asus, Tyan, or SuperMicro machines stuffed full of memory, with hard drives big enough to hold the OS plus an image of whatever they store in memory (ramdrives?), they'd require at most 3Gb (OS) + 4Gb (ramdrive backup). I don't recall seeing dual CPU's, but we'll go with that assumption.
The nice base machine we had settled on for quite a while was the Asus 1400r, which consisted of dual 1.4Ghz PIII's, 2Gb RAM, and 20Gb and 200Gb hard drives. Our cost was roughly $1500. They'd lower the drive cost, but incrase the memory cost, so they'd probably cost about $1700, but I'm sure Google got better pricing, buying the quantity they were getting.
The count of 88 machines per rack is a bit high. You get 80u's per standard rack, but you can't stuff it full of machines, unless you get very creative. I'd suspect they have 2 switches, and a few power management units per rack. The APC's we use take 8 machines per unit, and are 1u tall. There are other power management units, that don't take up rack space, which they may be using, but only the folks at Google really know.
Assuming the maximum density, and equipment that was available as "commodity" equipment at the time, they'd have 2 Cisco 2948's and 78 servers per rack.
$1700 * 78 (servers)
+
$3000 * 2 (switches)
+
$1000 (power management)
--------
$139,600 per rack (78 servers)
Lets not forget core networking equipment. That's worth a few bucks.
Each set of 39 servers would probably be connected to their routers via GigE fiber (I couldn't imageine them using 100baseT for this) Right now we're guestimating 1700 racks. They have locations in 3 cities, so we'll assume they have at least 9 routers. They'd probably use Cisco 12000's, or something along that line. Checking eBay, you can get a nice Cisco 12008 for just $27,000, but that's the smaller one. I've toured a few places who had them, and pointed at them citing them to be just over $1,000,000.
So....
$250,000,000 (ttl expenses)
- $ 9,000,000 (routers)
------
$241,000,000
/ $ 139,600
------
1726 racks
* 78 (machines per rack)
------
134,682 machines
Google has a couple thousand employees, but we've found that our servers make *VERY* nice workstations too.
I believe this to be a more fair estimate, than the story gave. They're quoting pricing for a nice fast *CURRENT* machine, but Google has said before that they buy commodity machines. They do like we do. We buy cheap (relatively) and lots of them, just like Google does. We didn't pattern ourselves after Google, we made this decision long before Google even existed.
When *WE* decided to go this router, we looked at many options. The "provider" we had, before we went on our own, leasing space and bandwidth directly from Tier 1 providers, opted for the monolythic sy
Serious? Seriousness is well above my pay grade.
> After many scientific and time consuming
> experiments, we have found the number of servers
> to be.........
> 42.
Yes, but each server is the size of a planet.
You might also be interested to know that there are a lot of government buildings in Washington DC.
Ha! It is quite a neat place. One of the nicest datacenters I've been in. They do try to take some precautions I guess. For example, you won't find the addresses to any of their datacenters on their website and the buildings are completely unmarked. They're usually not in business parks or anywhere you'd really expect them to be either. However, I'm sure that information could be dug up rather easily :)..
Now would you have to get into the datacenter first? Or are the concrete walls a non-issue?
-JD-
I think they include infrastructure and air cooling into their $250M figure. I these things can actually cost MORE than the racks themselves, especially if these racks consist of commodity hardware, and considering the size of their data center.
You just trolled yourself.
wasn't there recenly an article here on /. about how, due to the unreliability of mechanical/magnetic drives, it was cheaper (and faster, of course) for them to use solid state drives? i know i read that about some company. i couldn't find the article, though.
does it run Linux?
-------
Chunky Bacon
Based on that assumption, if they do have 100k servers and if those servers were each 1ghz wouldnt they be bigger than any supercomputer in terms of pure performance?
100k Ghz vs. VA Tech's 4400 Ghz.
Or am I completely wrong?
-------
artlu.net
Google "have?"
What is with this movement to consider corporations as plural rather than singular?
A corporation, i.e., Google, is SINGULAR. As in, "Google HAS pulled its machines out of a datacenter..."
If you said something like, "Google employees HAVE pulled their machines out of a datacenter..." then that would be correct.
If you're going to be pedantic, so am I.
If you read (actually, watch) this article, you'll find google was using 15,000 linux servers as of the year 2002....
"how google does it"
They use plutonium to power their servers.
They also have lightning rods all around their server farms to harness bolts of lightning if and when they do strike.
/^[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4}$/i
It would not be a very distributed DDOS and that would stop any attack quite quickly. Quite simply google's bandwidth providers (or the providers above them) would just unplug them. They may be global, but they probably have less than 40 datacenters. It would not be distributed enough to sufficiently attack. If you could take over the same number of machines with the same amount of bandwidth, but distributed globally on various subnets (say a massive virus), *then* you'd have a DDOS machine. As is, google's DDOS would be shut down quite quickly.
Photos.
If the numbers floated here of about 100,000 unreliable PCs, doesn't this make Google a serious industrial toxic waste hazard? There's a lot of lead and other exotic elements in all of these machines put together. Especially if they die quickly.
I am half kidding here but it still makes for a large amount of hardware to dispose at one point.
Getting into datacenters is trivial: Set up a shell company, and call the sales team and insist on inspecting their colo before comitting to anything. You'll get a guided tour of any hosting facility you want. Security at most hosting facilities is a joke - the CTO at a company I worked for used to spend quite a lot of time tracking down reliable hosting companies to use for our systems, and more than one place maintenance doors had been left unlooked etc.
Is there any correlation between the amount of FUD, disinformation and general rumer mongering that has recently exploded around Google and the fact that Microsoft is now trying to play their game?
Are we watching an attempt to unlevel the playing field?
Surely all this can't be due to Google's recent move to go public.
I would also like to take this opportunity to call SILIZIUMM a jackass.
When a company incorporates, the ownly people 'protected' from suit are the stockholders. Their liability is limited to their investment. CEO's, board members, and employees can most certainly be sued if they're involved. This is why companies have so many policies and rules, it's so that if anything happens they can say 'it wasn't our fault, our employee violated orders'.
Most lawsuits focus on the company, however, because they're considered responsible for the employee, and the fact that the corporation has loads of money where the employee/s in question don't.
I don't read AC A human right
I'm not saying that it's impossible. I'm sure any dedicated individual could do it. However, tours in datacenters are typically guided (especially at equinix). As far as getting in via unlocked doors, I'd say definitely would not happen here. You have to go through about 4 doors and 4 hand scanners to get in. There are no other entrances.
:). However, I'm sure it impresses many decision makers.
Of course, most of it is more for show than practicality. I mean, they have hand scanners on every single cage. Definitely a little bit excessive
-JD-
Who cares, MAE East and MAE West are MUCH more critical to the functioning of the internet and their position has been known for years. MAE East is on the other side of a wall from a parking deck, if you wanted to do a decent job of crippling the internet a truck bomb on the other side of that wall would do a pretty good job. Of course it would have been better before private peering facilities decentralized a lot of cross domain traffic but it would still cause enough traffic to be screwed up that it would have a very serious impact.
There are 4 boxes to use in the defense of liberty: soap, ballot, jury, ammo. Use in that order. Starting now.
Depends on if you're using IBM drives or not.
Our IBM SAN has a dead drive more often than not. We've gone from using RAID5 in each chassis, to RAID5 + 2 hot spares because we've had multiple failures (took out the entire chassis and made some customers very angry)
They're very smart offering the 1Gb storage.
I'm not sure if there's an offical standard, but generally a lowercase "b" is shorthand for "bit", while an uppercase "B" stands for "byte". The difference is almost an order of magnitude.
Google is offering 1 GB of storage, that is, 1 GigaByte, or roughly 1024 MB.
Many networks now use 1 Gb Ethernet, that is, 1 GigaBit per second, or roughtly 128 MB per second.
Then there's the whole MiB vs MB and GiB vs GB issue... (because 1000 != 1024) ((2^10 = 1024, 10^3 = 1000))
Google found the full text of the REAL ownership papers withe their caching robot, and SCO know. After all, it's Googles job to find obscure information.
Yay me!
The Secret Source of Google's Power
shut up, you fucking retard.
Google is dropping some serious $$$ on large system(s) here. Why did they choose to use clusters of PCs? It's got to be a nightmare to manage. Why not go for something from a company like SGI or IBM where you can get very large NUMA systems?
see title.
Do they have concrete walls protecting the cabling from sabotage as it leaves the building? What about 50 ft or 100 ft or 500 ft away? I'm sure a backhoe can be rented cheaply enough...
Some of the reasons these techniques aren't used in enterprise computing:
Since I've seen it up close a few times, I can say that the standard "enterprise way" (Oracle/Sun/EMC) delivers very poor bang for the buck. If Google wanted to, they could deliver a modified GFS with any desired level of reliability by increasing the redundancy. And even after that bloating, it would still deliver greater bang for the buck than the conventional solutions.
The high-end Sun machines are designed for high availability. Not only will a CPU failure not crash the machine, the CPUs are hot swappable so you can replace a failed CPU without so much as a reboot.
Yes, 10 years ago this was a important thing to have... As were many other "big iron" features. And it still sounds very cool in a geeky kinda way.
But with redundant relatively cheap clusters available, these types of things aren't worth the $$$ they used to be.
Except at the extreme high end of the computing world hardware is steadily progressing to commodity level.
Some people are like slinkies--basically useless but they bring a smile to your face when pushed down the stairs.
Yes, but you'd be surprised how many mid-level datacenters rely on security through obscurity. I know a semi-major datacenter in downtown Unnamed where you can walk into a large local landmark, go up in an elevator, and be one locked door from millions of dollars of equipment. I also know other datacenters in the same unnamed city with armed guards, biometric handscanners, and ID checks.. choose your datacenter wisely!!
Randall: In a row?
I hold it, that a little rebellion, now and then, is a good thing. -- Thomas Jefferson
Have a look at Rackable. Notice that Google is one of their customers. They offer up to 88 half-depth 1U servers in a rack and can even do DC power to cut down on heat generation from such a dense configuration. I'm sure Google is taking advantage of their custom configuration services, too.
While thinking philosophically, we see problems in places where there are none. -Wittgenstein
Thank you.
G
Do these machines all have real IP addresses, or do all but the front-end webservers have non-routable addresses?
If nothing else, keeping track of 100K addresses is pretty amazing.
Is this one of their machines? Google Search Appliance I wonder if all of them have that fancy yellow paint job?
Comment removed based on user account deletion
If I were Google, I would probably buy out space that I could move into later. But that is me.
^byrne :/
The exact number of servers that Google uses is trivially easy...
2,718,281,828
; )
You are SO full of horseshit. I checked out your little site and all I see is thinly veiled bigotry. The biggest problem is that a lot of dinosaurs here in the United States are trying to push things back to the 50s in terms of "values" and the economy. Hence the operations in Iraq instead of Afghanistan where they should have been. Face it. The United States had it's time in the sun during the 50s and now it's OVER. It's time for us to accept our place and rather being citizens of the United States, we must become citizens of Earth. I'll bet your grandfather knew more about being a citizen of Earth than you do. Fucking get over it it.
Remember: Language and grammar; or, Kelsey Grammer.
"I pass by their cages every time I go to my cage there."
Does anyone else find this funny?
is you crazy? that is a lot for a cheapo commodity server. I am starting up a search engine and I spend an average of about $300 for a server. Bang for the buck is what it's all about. Ok, even with the dual action going on, I can put together a 1u rack system with dual 2ghz xeons, 2gb RAM, and 2 160gb drives for under $1100. And I bet google can get stuff way cheaper than I can. fyi, that is using an Intel motherboard, samsung memory, and maxtor drives, so it isn't even the worst stuff you can buy.
FedEx can't just lose a few packages there and there.
Yes they can. They did for me, anyway. When I went through their claim procedure it was denied. It took a lawsuit, a couple of court visits, and a year later to get my money.
Apparently, they claim it was a communication problem - apparently there are 4 Fed Ex companies, and somehow, my complaint got to the wrong one. I don't know how that was my fault (as they insinuated), though, as they have only 1 claim reporting number.