Slashdot Mirror
MIT May Have Just Solved All Your Data Center Network Lag Issues
alphadogg (971356) writes A group of MIT researchers say they've invented a new technology that should all but eliminate queue length in data center networking. The technology will be fully described in a paper presented at the annual conference of the ACM Special Interest Group on Data Communication. According to MIT, the paper will detail a system — dubbed Fastpass — that uses a centralized arbiter to analyze network traffic holistically and make routing decisions based on that analysis, in contrast to the more decentralized protocols common today. Experimentation done in Facebook data centers shows that a Fastpass arbiter with just eight cores can be used to manage a network transmitting 2.2 terabits of data per second, according to the researchers.
A link to the paper is in the first article link. Direct link Here. They also have a GIT repo to clone, if you're interested.
This is a really bad idea. No need to elaborate further.
Most ACs are not even worth the keystrokes to insult them. Be generically insulted by this and ignored otherwise.
Your 300 x 10GB ports on 50 Servers is ... not efficient. Additionally, you're not likely saturating your 60GB off a single server, and you're running those six 10GB connections per server to try to eliminate other issues you have, without understanding them. You're speed issues are elsewhere (likely SAN or Database .. or both), and not in the 50 servers. In fact, you might be exasperating the problem.
BTW, our data center core is running twin 40GB connections for 80 GB total network load, but were not really seeing anything using 10GB off a single node yet, except the SAN. Our Metro Area Network links are is being upgraded to 10GB as we speak. The "network is slow" is not really an option.
Agent K: A *person* is smart. People are dumb, stupid, panicky animals, and you know it.
FTA: “This paper is not intended to show that you can build this in the world’s largest data centers today,” said Balakrishnan. “But the question as to whether a more scalable centralized system can be built, we think the answer is yes.”
Where in the comment did you read anything that suggested it would be about licensing? Or were you unaware that Berkeley and MIT are actual, real-world institutions, and it's possible to use those names without necessarily referring to the corresponding open source license.
File under 'M' for 'Manic ranting'
Maybe because that is what Token Ring did! Just sayin'!
"My immediate reaction is "WTF? What kind of moron doesn't make things 64-bit safe to begin with?" Linus
In fact, you might be exasperating the problem.
I hate it when my problems get angry, it usually just exacerbates things.
Nearly any network tech should be faster than Ethernet in certain circumstances. Ethernet is generally good though and appears to be quite good a scaling.
The key word, there, is scaling.
It looks like this is meant to make the network more efficient within a data center that handles a high volume of traffic, including high traffic spikes, by receiving a network time slot request from the end point (i.e. software running on a UNIX server) and sending a response that schedules packets to arrive just-in-time along a specific path to avoid queuing.
However, there is a less complicated way of achieving the same goal: Scalability - Increase your switch and server up-link bandwidth to eliminate congestion and queuing.
Yes, it costs money to add network capacity. But the big question is which would cost more? Adding capacity? or installing a pair of servers, rolling out software clients to all of your endpoints (servers), and supporting the system? Personally, I'd rather add network capacity and be done...
...are they trying to say that "Arbiter macht frei"?
This is about zero in-plane queuing, not zero queuing. There is still a queue on each host, the advantage of this approach is obvious to anyone with knowledge of network theory (ie. not you). Once a packet enters an ethernet forwarding domain, there is very little you can do to re-order or cancel it. If you instead only send from a host when there is an uncongested path through the forwarding domain, you can reorder packets before they are sent, which allows, for example, to insert high-priority packets into the front of the queue, and bucket low priority traffic until there is a lull in the network.
Bandwidth is always limited at the highend. Technology and cost always limits the peak throughput of a fully cross-connected forwarding domain. That's why the entire internet isn't a 2 Billion way crossbar switch.
Furthermore, you can't install 6x 10-gigabit ports in a typical server, they just don't have that much PCIe bandwidth. You might also want to look at how much a 300 port 10GigE non-blocking switch really costs, multiply that up by 1000x to see how much it would cost Facebook to have a 300k node DC with those, and start to appreciate why they are looking at software approaches to optimise the bandwidth and latency of their networks with resources that are cost-effective, considering their network loads like everyone else's network loads never look like the theoretical worst-case of every node transmitting continuously to random other nodes.
Real network loads have shapes, and if you are able to understand those shapes, you can make considerable cost savings. It's called engineering, specifically traffic engineering.
-puddingpimp
Nah. They put MPLS logic-- deterministic routing by knowing the domain into an algorithm that optimizes time slots, too.
All the hosts are know, their time costs, and how much crap they jam into wires. It's pretty simple to typify what's going on, and where the packet parking lots are. If you have sufficient paths and bandwidth in and among the hosts, you resolve the bottlenecks.
This only works, however, if and when the domain of hosts has sufficient aggregate resources in terms of path availability among the hosts. Otherwise, it's the classic crossbar problem looking for a spot marked ooops, my algorithm falls apart when all paths are occupied.
Certainly it's nice to optimize and there's plenty of room for algorithms that know how to sieve the traffic. But traffic is random, and pathways limited. Defying the laws of physics will be difficult unless you control congestion in aggregate from applications where you can make the application become predictable. Only then, or you have a crossbar matrix, will there be no congestion. For any questions on this, look to the Van Jacobsen algorithms and what the telcos had to figure out, eons ago.
---- Teach Peace. It's Cheaper Than War.
Your 300 x 10GB ports on 50 Servers is ... not efficient. Additionally, you're not likely saturating your 60GB off a single server,
It's not so hard to get 50 gigabits off a heavily consolidated server under normal conditions; throw some storage intensive workloads at it, perhaps some MongoDB instances and a whole variety of highly-demanded odds and ends, .....
If you ever saturate any of the links on the server then it's kind of an error: in critical application network design, a core link within your network being saturated for 15 seconds due to some internal demand burst that was not appropriately designed for is potentially a "you get fired or placed on the s***** list immediately after the post-mortem" kind of mistake. Leaf and spine fabrics which are unsaturatable, except at the edge ports: are definitely a great strategy to approach sizing of core infrastructure --- from there most internal bandwidth risk can be alleviated by shifting workloads around.
Latency performance seriously suffers instability at ~60% or higher utilization, so for latency-sensitive applications especially: it would be a major mistake to provision only enough capacity to avoid saturation, when micro "bursts" in bandwidth usage are the reality for real-world workloads.
An internal link with peak usage of 40% or higher should be considered in need of being relieved, and a link utilized 50% or higher should be considered seriously congested.