Slashdot Mirror
How To Build a 100,000-Port Ethernet Switch
BobB-nw writes "University of California at San Diego researchers Tuesday are presenting a paper (PDF) describing software that they say could make data center networks massively scalable. The researchers say their PortLand software will enable Layer 2 data center network fabrics scalable to 100,000 ports and beyond; they have a prototype running at the school's Department of Computer Science and Engineering's Jacobs School of Engineering. 'With PortLand, we came up with a set of algorithms and protocols that combine the best of layer 2 and layer 3 network fabrics,' said Amin Vahdat, a computer science professor at UC San Diego. 'Today, the largest data centers contain over 100,000 servers. Ideally, we would like to have the flexibility to run any application on any server while minimizing the amount of required network configuration and state... We are working toward a network that administrators can think of as one massive 100,000-port switch seamlessly serving over one million virtual endpoints.'"
I hope they have invented something better than ordinary Ethernet cables to wire that ting with.
http://www.intellipool.se/ - Intellipool Network Monitor
I have nightmarish pictures popping into my head of a waterfall of ethernet cables spewing from this with user's ports un-numbered with no network diagrams. People bashing on the server room door in a zombie like state muttering "MRRRHH FACEBOOK!" "TWWIIIITEEEuggggghh" with me inside screeching "NO! NO! I DONT KNOW WHAT PORT YOUR DESK IS! NO! I CAN'T MAKE THINGS GO FASTER!" before curling up in a ball listening to the hum of servers and the lamentations of the users outside the door desperately scratching to get in.
Be you Admins? nay, we are but lusers!
I've long been of the opinion that putting more than a few hundred hosts on a single layer 2 network is almost always a bad idea.
What do you do about broadcast storms? How do you prevent some clown from anywhere in that 100,000 machine cloud from poaching another machine's IP address (either maliciously or by an accidental typo)?
Subnets and routers were invented for a reason. Just because you can bridge the whole world together into one massive virtual Ethernet segment doesn't mean you should.
The paper is about adding a layer of addressing so that IP and Ethernet addresses can be moved from one machine to another as instances of virtual machines are migrated around. It's not about the problems of physically building a very large switch. The switch components are mostly stock items.
Have fun replacing it when it fails. In my head I imagine something like this.
Lets see... That's 100,000 ports with 2 LEDs each (link, action/fdx/speed/poe) for a total of 200,000 LEDs. Lets say they use some of the cheapest SMD LEDs on the market. Well use digikey part number 160-1183-1-ND which is a cheap 0603 foot print green LED. At quantity 200,000 that comes out to $12,000 in cut-tape packaging or $9,450 if you buy 210,000 of them in 3,000-qty reels.
Lets say that all of the link LEDs are on 100% of the time and the the activity LED is on 50% of the time. That gives us 150,000 LEDs on at any given point in time. Our example LEDs use 20ma at 2.1V. So 150,000 LEDs at 20ma uses 3Ka. In total, 6.1Kw is burned by the green LEDs.
All that blinking... Damn. I want one NOW!!! More than a girl friend!
I can't just go out and buy 33,334 d-links and turn off DHCP on all but one of them?
For justice, we must go to Don Corleone
Without getting too far into it, their brilliant plan to to insinuate a layer 2 and a half using "pseudo MAC addresses," using a directory service rather than broadcasts. They're hoping they can use this mess to paper over horrific network design.
Yeah, I'll grant you you might be able to cobble this mess together in an academic setting, and sure, you'll even be able to rig some demos that show miraculous increases in speed.
I can guarantee they'll find funding with their promise you'll even able to hire even LESS skilled network admins, meaning Zaboomafoo the Typing Lemur now has a shot at his CCIE.
But, damn, you ignorant twits. Most corporate networks are already mashed together by the most cut-rate cable monkeys they can find. The last thing we need is some half-assed "protocol" that will guarantee even more network designs that are guaranteed to trip and break their necks over the first packet.
He put his boots up on the table and made a face. "The sig," he smirked. "You can waste your life in search of the sig."
This seems to be a solution to a nonexistent problem. A big router, for example a cisco CRS, can be a single node supporting any data center. And it is a router, so there is no need for any exotic solution (L3 inspection on a switch?). It has a max bandwidth of 80Tb/s or 80,000 Gb Ethernet nodes. The beauty is of course that you can configure your entire data center with a single router, which greatly simplifies the network configuration, and makes changes simple.
don't cut it off www.mgmbill.org
They're basically NATting the layer two protocols. Combined with a super spanning tree for the natted addresses they're practically boosting layer two into layer three.
Before I read the paper I was thinking that it would be easier to just run all your services NATted at layer three, even using something like PPPoE (which is how cable networks solve the same basic problem, with something like half a million end-points on the same subnet). I guess it's more efficient to work with the simpler layer two protocols instead.
I would seriously hate to be the guy that tripped over that power cable.
A sentry gun will be installed in the power cable corridor, to execute you the precise moment you've done your tripping. So you wouldn't have time to hate being yourself.
(redundant, redundant power. I know, I know)
To answer your worried look: yes, there's a redundant sentry gun for the other cable too.
... they have only needed 1 port! :)