Slashdot Mirror
Google Pulling Back the Veil On Its Custom-Built Data Centers
jfruh writes: In the mid-'00s, as Google scaled up its data centers to meet increasing demand, "we could not buy, for any price, a data-center network that would meet the requirements of our distributed systems," says Amin Vahdat, the company's networking technical lead. So they had to build their own software-defined networks inside what were essentially vast warehouse-sized computers. And now the company is starting to tell the world how they did it.
Right now there's a massive push for even the smallest web projects to be "cloud scale" to this degree. No, the crazy custom shit that Google does in their datacenter is not something you need in YOUR datacenter.
Sooner or later Dice is going to render this site unusable. What a pack of fucking retards.
The world's burning. Moped Jesus spotted on I50. Details at 11.
I think you give them too much credits; they've been trying for years and haven't yet succeeded :)
Seriously. I'm building my new data warehouse inside a wooden footlocker. All I need is an ATX extender and to finish building the drive frames (for 8x3.5" drives, 12x2.5" drives and two DVD burners), and the back arm for the VESA mount for the monitor, then it all gets bolted together and fired up. It looks fuckin' sweet.
Political debates have me rolling my eyes so much I think I got optical whiplash. I should sue. - Foamy The Squirrel
What the hell is with the stupid share thing? When I point at it, it looks like they are trying to tweet "F-ing"!
Why the fuck are you fucking around with the /. UI? Do you have to change things for change's sake? If it works, stop changing it!
I promise I won't accidentally share a story over social media instead of clicking "Read more", no dice Dice.
In other words, mindless rubes and hipsters. Soon they'll rename Slashdot iDot or iSlash, of iTwitfuckhamshitprick. It's going to get so bad even APK is going to walk, and then where will I get my hosts file from?
The world's burning. Moped Jesus spotted on I50. Details at 11.
Slashdot also pulling back the veil on its crappy share buttons.
That is exaggeration, they simply want /. to be non-tech site where people don't know any better and don't block ads and over-share on social media. Too bad, they should have done their homework before buying into this community.
So, how many Libraries of Congress is that anyways? ... oh wait ... the Google blog post (ya'know, the actual artist, not the article talking about the article which was linked from the summary) actually states!
"Our current generation — Jupiter fabrics — can deliver more than 1 Petabit/sec of total bisection bandwidth. To put this in perspective, such capacity would be enough for 100,000 servers to exchange information at 10Gb/s each, enough to read the entire scanned contents of the Library of Congress in less than 1/10th of a second." = Source: http://googlecloudplatform.blo...
Lighten up, Francis. To normal people "any price" translates to "any justifiable price", at least in this usage.
Well, if you have a lot of awareness of the landscape, you can sort of fill in the pieces. Basically what they did was reveal enough to show that they may be clever, (the principles described are sound) but not enough to actually change the landscape. What they purport to have done is already an obvious high level strategy to anyone in the world who would have actually been able to pull it off and probably is already in play at some of their competitors. If you ask someone who actually recognizes the specific cited components, they can testify that it's a reasonable result though they are likely unable to say they could trivially reproduce the success. I would say the nearest competitor to Google with external evidence that they *could* possibly do this is IBM (their POWER servers used to support a proprietary interconnect that acted like this in the super computer space), though I don't know how much of that team is left or how capable IBM is at actually gathering those folks together for their cloud initiative.
The short of it is they used OpenFlow to enable them to construct an ethernet fabric that resembles a supercomputing fabric. I thought from the first time that I got a high level briefing on OpenFlow that the openflow controller role was quite reminiscent of the subnet manager concept that is ubiquitous to high speed fabrics (today basically Infiniband, but other fabrics historically did the same thing). By using OpenFlow, they get out of the trap of having to use equipment from the dwindling number of suppliers (if you want to buy a new fabric today, basically only three companies do it anymore, Mellanox, Intel, and Cray). They don't get *all* of the benefit of those fabrics, but they get the vast majority of the benefits that would be relevant to the Google workload, at siginificant reduced cost. Basically, in 'traditional' ethernet, there's a gread deal of gymnastics to have a loop-free network from the perspective of any particular participant while at the same time doing all sorts of things with aggregation and MSTP and other technologies to gain some benefit. In a supercomputing fabric, they go to town with mesh networking that doesn't give one crap that there are 'loops' in the network because the entire route of each packet is settled at the edge.
There are things in a modern Supercomputing fabric that I don't see mapping to OpenFlow, but I could see how a sufficiently invested company could negate the need of some of it in something designed particularly for their application.
XML is like violence. If it doesn't solve the problem, use more.