Slashdot Mirror
Inside the Google-Plex
tappytibbins writes "Baseline magazine has an in-depth story about how Google manages its own IT infrastructure. From the article: 'In general, Google has a split personality when it comes to questions about its back-end systems. To the media, its answer is, "Sorry, we don't talk about our infrastructure." Yet, Google engineers crack the door open wider when addressing computer science audiences, such as rooms full of graduate students whom it is interested in recruiting.'"
print friendly version, because their page layout is a little too far on the "hey, if we add more adverts, we'll make more money!!1!"/WiReD-more-colors-are-good end of the scale.
I want inside the google party plane!!
Take the cheese to sickbay, the doctor should see it as soon as possible - B'Elanna Torres, "Learning Curve"
I'm still waiting for pictures of the "party plane", though.
What I'm listening to now on Pandora...
Its nice to know that some companies are willing to open their doors to the Tech comunity. Reminds me of Open Source Software... but only with hardware
It still worries me that google will soon know everything about everyone. I hope they dont share that data with ANYONE.
Mod others as you would have them mod you.
I feel bad for the sysadmin
TOP DSLR Cameras Reviews of the top DSLRs
Looks like it's time for Google to start making prospective students sign non-disclosure agreements...
Currently hooked on AMP
Most journalists and business analysts are notable for doing a half-assed job and taking credit for cut & paste jobs. Journalists who actually spend time researching their stories are a dying breed, so my take on this is that Google would rather not waste their time answering stupid questions from people who don't even understand what they're publishing. Their time is much better spent recruiting smart people or just talking to grad students in some sort of academic goodwill.
An alum of my university who works at Google recently visited and gave an informative lecture with a long Q&A session. I can vouch for the fact that we were told more than I've ever been able to read online about the way Google manages various issues, like their IT infrastucture. However there were still limitations to what he would/could tell us (sorry I won't go into specifics). It seemed (as you would expect) the better our questions, the better his answers, and if we asked questions that were too good, then it was likely that he did not feel liberated to answer.
Also, Google was cool enough to sponsor a Programming Contest and a Graduate Research Conference we held. Our alum attended our little conference and had great feedback and questions for our presenting students. With respect to knowledge, intelligence, and humor this guy was all I would imagine and/or hope for one of our alums working at Google.
On the otherhand, I was very unimpressed with certain issues concerning lack of professionalism in the lecture. As one example, though this is only an impression, it seemed that he felt he could just get away with wearing jeans and a Google t-shirt for the few days that he was with us because he worked at the ever prestigious Google. It seemed a bit arrogant. Also keep in mind that his position at google is higher than a solutions engineer.
Just thought I'd share.
Falun Dafa is good!
...that a single search engine runs on hundreds of servers. What do they need all of those machines for?
My journal: Clicky. Read it because it
If it is anything like their web-presence, half the stuff must have 'Beta' appended to it.
New GPayRoll-Beta!
If this signature is witty enough, maybe somebody will like me.
They have two, you insensitive clod!
They still haven't been able to break that 225000 to 1 host to sysadmin ratio barrier yet. But they're working on it.
Give a man a fish and you have fed him for today. Teach a man to fish, and he'll say "WHERE'S MY FISH, YOU IDIOT?"
I was interested in working for Google -- mostly for a job. I even had someone from inside recommend me -- I figured I'd be in, no problem. Rather than being interviewed for my skills, or the relevant department, they interviewed me for a sysadmin. No problem, I'm a sysadmin I thought. I didn't do the CS route at a university, though, and there were some highly relevant things that I just didn't know how to answer. I didn't pass round two.
So I tried to get another interview for a while, but no bites. Google has made it clear that they aren't interested in my work. I've stopped trying to get a job there. Besides, I don't think I can sit in a cube and take long drudgery with the occasional stinging bits of punnishment. I like all my punnishment unending and all at once, and so I just work for startups.
Zhrodague.net - I do projects and stuff too.
Ziff Davis sites are always chock full of adverts, they are just heavy duty web spammers who care more about adverts than content, the spammers spammer
why slashdot links to them is anyones guess, the articles are always light on content and heavy on the adverts, not that anyone here cares as we are all running adblockplus, firefox and a hosts file
So the question to ask is: Who is `we'? I could provide an answer. `We' here, is Google's official answer to such questions from the ever inquisitive press men. Those who speak on behalf of Google have been asked to memorize that answer if they do not wish to talk about a topic. It works well. This approach reminds me of the government's `We can neither confirm nor deny...' mantra.
What follows next could be interpreted as a Google engineer's answer to a question, which answer may simply represent one of many possible implementations and NOT one you could find supporting Googles infrastructure.
Guys, this is all about semantics and context. Good night!
I don't know what Google's dress code is, but I do know that when, say, Oracle sends a consultant out to help with setpup/problem resolution, etc. they usually show up in business-casual attire - khakis and a button-down shirt with their Logo.
Again, I don't know what Google's stance is on dress. They may be perfectly fine with Jeans and a T-Shirt.
Media today would not have a use for such mumbo jumbo. I mean really, what reporter is going to cover a topic on computer infrastructure for a search engine? You see what they did with the story on Google with China. I mean, what percentage of the population would even be interested in a topic like that, let alone understand it, unless it was brought out in a political light. They only cover issues that are going to sell. You can't listen to the news for actual intellectual information. Media has really lost a lot of integrity. Lets hope Google can help give that back to people through online media. There are so many times that I think to myself, why do computer geeks make everything so much more difficult than it really is. So that other people won't try it for themselves? It's a constant process anyway. The fact that they are being so secretive about it just tells me that it most likely is based on theory in business, not academia. They are in a very dangerous position. It's more than likely they run across more confidential information than one could imagine. I would suspect that they have to put much effort into placing utmost importance on information security. One thing I will agree with, Google made a wise political decision with China. After all, what better way to keep afloat than by making a country that is able to obtain credit in the global economy your friend. Lets hope they are pulling that credit from our direction, they've overinvested to the point that they may need it soon. If people can't see past who is who in this world and see what is really going on, then I feel that they should inform themselves. Google it. Better yet read a bit.
In other news, Google has started tagging new employees on probation with 'BETA' labels.
Help a man when he is in trouble and he will remember you when he is in trouble again.
Everyone is talking about GoogleFS. But no one is talking about how they manage structured data. How do they do it? Some SQL stuff, some homegrow potion, or have they managed to create a sensible interface for structured data on top of GoogleFS?
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Lin
This is probably just another Google Publicity Stunt. I've worked at several of the top companies, and its just "people" that work there, not much more "intelligent" or better than others who choose a lower profile, more fun and probably more lucrative environment to work in. Who has any information to prove it otherwise?
mb
Here are some good papers about Google's technologies:
Sawzall (simplified scripting on top of MapReduce)
MapReduce (Google's massively parallel system based on the concept found in functional programming. The system takes care of managing jobs, parallelism, and fault tolerance, allowing engineers to more quickly produce code.)
GFS (Google's File System)
Google's Cluster (An older paper describing how Google's search cluster works. The cluster described in this paper is a few generations out of date.)
BigTable (Google's semi-structured database. There haven't been any papers released, but this is my write up based on a talk given in October 2005.)
And here are some videos:
The Google Linux Cluster. This is an older video where Urs Hoelzle talks about their system and focuses more on the hardware side of things.
Google: A Behind-the-scenes Look. Jeff Dean gives an overview of most of the technologies mentioned in papers above. I thought the demonstration of Google's internal word clustering was interesting (and funny).
Perspectives on the Information Industry. This is a technology-light (IIRC) talk given by Eric Schmidt.
BigTable: A Distributed Structured Storage System. The talk from which I created my BigTables notes (above).
Andrew
sounds like something that would go on in a party plane.
Hawt!
An Inside Look at Google
;-)
:-p
Hey guys, watch it, it's about the women at Google!
I guess now we have yet another reason to go there.
Beware: In C++, your friends can see your privates!
Arrogant? Arrogance would be whipping his thingy out and pissing on the front row.
Considering that Google has 450,000 servers, that would be one massive e-penis, from which I would gladly accept the gUrine.
I am defenseless. Use your button. Mod me down with all of your hatred.
However there were still limitations to what he would/could tell us (sorry I won't go into specifics).
Funny, I didn't notice anyone ask.
They use Oracle Financials on your typical "five-9s" hardware.
:3
They had a big fight switching to that, before that they were using (drum roll please), QUICKEN!!! (lol)
Although the HR apps are custom written. Exciting
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
It's not the highly parallel clustered racks of custom-designed linux servers that makes Google Google. That's an enabling feature. Rather, it's their custom engineered application-level operating environment, if you will, which runs on top of that. It's good at keeping data, indexing attributes, finding it, breaking problems down, and intelligently routing work and results. The search engine and all their other apps are built on top of this, and it allows their engineers to leverage this common distributed platform in all of their external and internal applications.
=== End Elevator Summary ===
Not many companies are willing to write their own application layers to deploy services. Most companies CAN'T. It's just not worth it. It's worth it to Google because developing and deploying world-wide information retrieval services is their business.
However, a standardized Application OE that can run and take advantages of the resources of many potentially unreliable computing resources would be very valuable to many businesses.
Grid technologies, web services, J2EE, and clustering technologies are just scratching the surface.
THIS THING CAN TURN ON A DIME, MACROSSZERO STYLE ALSO FUCK BETA, ~NYORON
I went through the majority of the process (phone interviews, then the face time at the New York facility and a trip to California) before I told them I wasn't interested in the job. My reasons for turning them down were three. First, the 80/20 deal was a myth if you were going to pursue something they were really interested in. I wanted to complete my PhD. Now, this PhD was not in a field they cared much about (despite their glaring need of my skills for one particular service), so that part of the deal was mumbled everytime it was mention. Second, the pay wasn't good enough to basically live at the facility. Third, the interview process was abusive in many respects. The first phone call was with a guy consumed by asking me about my doctoral research and my knowledge of how inodes work. He kept shifting between the two. When I asked him why this was even necessary given the position I was applying for, he got irritated and said that you had to know the ins-and-outs of how a file system works in order to configure (something I wouldn't be doing) any part of their infrastructure.
This lead to my observation of part of their file storage system which is quite possibly the most tweaked NFS nightmare/genius/what-the-fuck I'd ever seen. My past experience with networked file system was, I admit, very limited compared to what they had going on. Now, again, I wasn't even going to have anything to do with this system or any sysadmin work at all, but it was obvious that they wanted you to at least have knowledge of the system on some level beyond the user. It also came across as a showing-off culture too. I am glad I didn't take the job for various reasons, but if you are a sysadmin freaker who loves dinking with shit, you'd fit in; especially if you like to show it off too. Just be prepared to have some middle manager there fuck with you for a hour or two on the phone before you get to the outer part of the inner sanctum.
Comparing it to Windows will be a moot point, since El Dorado is going to have a 40% larger code base than XP.
At the business end of the spectrum, you have the MBAs and the like, and the more expensive the suit you wear, the more respect you get. Pick top-level executives from any company other than Google and a very few other technical companies, and you'll see what I mean. They will always wear expensive suits, and people who aspire to be in their position will dress likewise. Here are some examples of the look that this culture aspires to. Searching for most of the names you'll find in Forbes magazine will come up with similar results.
At the technical end of the spectrum, you have hardcore techies who live, breath and sleep in the world of concepts and structures and mental things that the MBAs don't understand or care to understand. They're mostly extremely mental people, and they pay more attention to what's in their head than what's in the world around them. If they're successful (as determined by their peers), they are successful because of their skills, not because of what their peers think about their cloting, or because people are impressed with their $20,000 wardrobe and 7-series BMW. Here are some examples of a guy who epitomizes the jeans culture. Search for similarly revered technical people, and you'll come up with similar results.
As my professor explained, know your audience, and dress accordingly! Suits don't respect you if you wear jeans, and jeans don't respect you if you wear a suit.
Little in the way of structuring data
x .html
by leandrod (17766)
Everyone is talking about GoogleFS. But no one is talking about how they manage structured data. How do they do it? Some SQL stuff, some homegrow potion, or have they managed to create a sensible interface for structured data on top of GoogleFS?
--
Leandro Guimarães Faria Corcete DUTRA
DA, DBA, SysAdmin, Data Modeller
GNU Project, Debian GNU/Linux
----------8<----------
Wild Guess® ?
Rob Pike?
http://herpolhode.com/rob/
http://plan9.bell-labs.com/plan9/
http://plan9.bell-labs.com/wiki/plan9/Papers/inde
http://plan9.bell-labs.com/sys/doc/
http://www.vitanuova.com/inferno/
http://herpolhode.com/rob/hotchips.html
leandrod,
You might find this especially interesting considering your affiliation.
~hylas
Every second week we get a new story that tells us how awesomely secret Google is, how they never tell us anything, and how they have many, many PCs.
Granted this is great marketing, but it is pretty boring.
Google: n. (1) An Internet Search Engine. (2) Tried various other things. None exciting. Still, thought they try and claim otherwise, still an Internet Search Engine.
How is this for 'inside google' news?
d _plane/
http://www.theregister.co.uk/2006/07/07/google_be
Actually, new people at Google are known as Nooglers.
Software sucks. Open Source sucks less.
They sell Google-in-a-box appliances without root access or source. If it contains Red Hat Linux isn't it a violation of GPL?
i base my personal presentation for interviews on my experience in 1997 when i was 19 and got an interview with a waaay old school fortune 500 company. figuring that if i showed up in my dressiest clothes i would be expected to continue to dress that way, i showed up wearing the best clothes i could feel comfortable in -- jeans, a short sleeved tennis shirt and my always-on black zip-up hoodie over top. the interview was conducted by four people, one manager in a suit, two workers in business casual, and the most tech-knowledgeable in jeans and t-shirt. despite appearances, my professional demeanor and relevant knowledge convinced them that i was capable. i was later told my dress told them that i wasn't trying to hide anything. i got the job.
fear is the mind killer
What are Googleplex's similarities with the Borg cube? does it repair itself when a node goes down? is it possible to bring the whole system down by injecting a false command in a node? etc
Am I the only one who got the feeling that the entire article was one big babble about things everybody who reads any given daily newspapers knows? Amazing how one can craete an entire article that says absolutely nothing.
I wonder how this fast-food approach to hardware will behave over time. On one hand you have most of well established software companies that prefer stable (and expensive) hardware, but on the other hand you have this unique and very successful company that runs the "consume until it dies" pattern.
cut this signatures madness. stop reading them now!
"With his unruly hair dipping across his forehead, Douglas Merrill walks up to the lectern set up in a ballroom of the Arizona Biltmore Resort and Spa, looking like a slightly rumpled university professor about to start a lecture."
This sounds more like the beginning of some Harlequin romance novel than the first line of a serious piece of journalism. The sentence even manages to run-on and has a simile. I expect that by the end of the article his hairy chest will be bared by the protagonist and that they will live happily ever after.
What those who want activist courts fear is rule by the people.