Network Testbed Emulab.net

gseidman writes: "Have you ever needed to simulate a network? Tired of old ns? Do you just hate dealing with hardware in general? Take a peek at Utah's Emulab.Net. They have over 300 PCs, some StrongARM devices, roughly 5 miles of cabling, a huge and expensive switch, and great software for setting up a virtual LAN. They also have a gallery showing the machine room in various stages of completion (did I mention five miles of cabling?)."

Comments

This is very cool.

[perdida]

This is applied research, the type that may have direct and positive effects on improving security and efficiency in the immediate future.

Unlike dark matter research, Mars colonization, and subatomic research, this stuff is the kind of thing that should attract wide funding from business. Immediate payoffs are likely.

Basic research is fine, but I wish that the money poured into it would go towards immediate business applications. More available cash would make those venture capitalists a lot nicer and less demanding of unrealistic profits in an unrealistic period of time.

5 Miles of sloppy cabling...

[GLX]

I don't know what brainchild designed the layout of those switches, but the fact that they went for the RJ-48 blades versus the Telco (RJ21X) blades in those switches shows sloppy planning. What if one of the blades in the middle goes out? They have to unplug 48 Cat-5 cables versus unscrewing 4 telco harmonicas and leaving the 48 ports untouched on a patch panel

If you're going to put forth that kind of effort and money, why not do it right the first time?

Re:5 Miles of sloppy cabling...

[mac.newbold]

There are several reasons we chose not to use the Telco connectors:

1. Then we'd have to go to a patch panel, and then the ciscos, which greatly increases the number of wires and the time it takes to run them.
2. The telco connectors aren't as dense enough to be worth it... 8 wires per cat5, 50 wires per telco, so we'd only consolidate 6 wires at a time.
3. We eventually want to run gigabit over copper with these cat5e wires, and if we put them into telco connectors, we can't do that.

Does that answer your question?

Mac

Re:5 Miles of sloppy cabling...

[GLX]

Not especially.... To answer in order:

1) It increases the number of wires by 4 per 48. It however lessens the trouble involved if a module fails - apparently you've never had the joy of unplugging, keeping track of, and re-plugging in 48 Cat-5 cables in a hurry. It's not fun. An increase of time in the beginning far outweighs the risks of the increase of time in an outage. I can show you pictures of rats nests and tell you horror stories all day about this.

2) The standard Cat-5e configuration still only uses 4 wires. The Telco panels are wired as such. For each Telco harmonica you get 12 ports - quite dense enough.

3) There's nothing that says that gig won't be supported over telco, just like there's still no set-in-stone standard for gig over cat-5. Nothing even says that cat-5e is going to be required.

Agreed

[clark625]

I totally agree with you. Simulation doesn't prove that things will be all peachy cream later on in the "real world".

Then again, if a 1/100th scale model does a nose dive in the wind tunnel when they throw a little turbulence at it, you can bet the ranch that plane will be redesigned. Testing and modeling only show the existence or non-existence of very particular problems. We just usually hope that we can "test" out as many likely problems as possible.

Re:5 miles, big deal.

[mac.newbold]

I was one of the few 'lucky' people who had to run it, and no, we didn't run it thorough a 2ft high crawl space. It is even worse than that.

In case you couldn't tell from the pics, this is all in self contained racks. The large majority of the wiring is in 9 standard-sized racks, or about 7ft tall * 3ft deep * 2 ft wide * 9 racks = 378 cubic feet for about 5 miles (25,000 ft) of cable plus all the PCs and switches.
As a generous estimate, that leaves 100 cubic feet for cables and ventilation. That says the every cubic foot of open space is filled with an average of 250 linear feet.

Needless to say, it was not fun.

Mac

Re:This could be a nightmare

[mac.newbold]

5 miles of cables? I hope they labled them... :)

Every wire (~1000 of them) on both ends, then the endpoints get recorded in a database.

And anyone who moves a wire without properly documenting it gets shot! ;)

Mac

Only 5 miles?

[yzquxnet]

The post submitter makes 5 miles of cable seem like a lot. Well, it isn't. Even a small office complex can burn through 50,000 ft a cabling in short work. Running two data cables effectivly doubles your cable requirements. Depending on the situation the installer may even lay a third cable for voice. (What standard was that?) Anayways, 5 miles of cable is not a lot.

Re:logN cable

[Anonymous Coward]

The reason for the cables is that every machine has 5 Ethernet ports.

Notice the hardware config?!?

[Geek In Training]

Boy, I wonder if one of the projects they're planning to emaulate is "wide scale hardware failure." Look at their node configuration (hard drive boldfaced):

128 new nodes:
850Mhz P3
512M ECC memoryold reliable BX chipset
40G 7200rpm IDE disk (IBM Deskstar 60gxp)
5 Intel Pro/100+ network interfaces
2 on board
1 on a single Intel card
2 on a dual Intel card
No video at all
serial console

This is the very same hard drive drive we drew and quartered here, and has gotten IBM a big fat lawsuit for rampant failures.

So, I guess their error recovery is going to be tested to the limits very shortly, especially with the space/heat issues inherent in the installation exacerbating the engineering flaws in the 60GXPs. :)

Re:Notice the hardware config?!?

[A Commentor]

Wrong... that was the 75GXPs. See the original article.

I double checked a few posts, and they mentioned that the 60GXP was much more reliable.