A 50 Gbps Connection With Multipath TCP
First time accepted submitter Olivier Bonaventure writes "The TCP protocol is closely coupled with the underlying IP protocol. Once a TCP connection has been established through one IP address, the other packets of the connection must be sent from this address. This makes mobility and load balancing difficult. Multipath TCP is a new extension that solves these old problems by decoupling TCP from the underlying IP. A Multipath TCP connection can send packets over several interfaces/addresses simultaneously while remaining backward compatible with existing TCP applications. Multipath TCP has several use cases, including smartphones that can use both WiFi and 3G, or servers that can pool multiple high-speed interfaces. Christoph Paasch, Gregory Detal and their colleagues who develop the implementation of Multipath TCP in the Linux kernel have achieved 50 Gbps for a single TCP connection [note: link has source code and technical details] by pooling together six 10 Gbps interfaces."
Doesn't SCTP provide for these scenarios (and many more)?
Matt
RFC 6182 if anyone is interested.
-- I have a private email server in my basement.
yes, I know etherchannel load balancing ... but maybe that would be easier to "fix" than inventing something that mostly exists. ... 10 years ago.
without every user making 3 connects to view their friends cat picture.
yes, this could have some good uses, no, dont release it to the drooling masses so 12 year old Tiffany can stream Justin Bieber videos while texting her buddy sitting right next to her "faster".
I remember getting dual-channel ISDN, which was 128k, but it was split into two 56k data channels and a 16k control channel. You could never download from any one site faster than 56k because a connection couldn't straddle more than one data channel.
Still, I could play EQ and surf at the same time on a different computer, a novel thing you young punks take for granted get off my lawn!
(-1: Post disagrees with my already-settled worldview) is not a valid mod option.
Wouldn't six 10 gig connections add up to one 60 gig connection instead of one 50 gig connection?
One of the barriers to this technology will be API support. Many APIs provide the IP address (on both sides) with the connection object. Implementors will have to make a choice about which ip to expose and remain backward compatible.
If you want to use multiple links all at the same time, with the packets spread over them, you're supposed to get an Autonomous System number. You then get an entry in the routing tables, and the regular old IP packets flow over both links.
The trouble is that providers may refuse to keep you in the routing tables. This can be fixed, legislatively if need be. We may also need more AS numbers; they were recently changed from 16-bit to 32-bit. Perhaps we need 48-bit.
Aside from the troubles though, AS numbers are the way to go. They are the proper and correct solution to this problem.
I'm possibly missing the point here, but I'm struggling to understand how this would be put to use:
Servers that can pool multiple high-speed interfaces:
This capability has been around for years - port channeling/interface bonding/whatever vendors call it, do this already at L2. Aside from making it easier for those interfaces to sit on different subnets, why would you want to push this capability to L3/TCP? Seems like it just introduces additional complexity.
"Smart" devices (or any client really)
I suppose this makes a bit more sense, but given the order of magnitude latency and (typically) bandwidth differential between your typical 3/4G and a local wireless connection, this doesn't seem like a huge boost - 3/4G in particular is more hampered by latency than net available bandwidth anyway. Does a wider pipe really boost things that much?
On a side note - I thought TCP was all about guaranteed and ordered delivery - aren't you just stuck waiting for the slowest link in the chain anyway?
Their use case uses all interfaces on a machine, then they disable some of them. Do you normally WANT to use every and all interface for your traffic? If you're a mobile device, maybe not?
I want to delete my account but Slashdot doesn't allow it.
I understand the application sets up its sockets as usual, and the kernel add TCP extensions for MTCP, and use them to negociate with remote endpoint and startup the subflows. This is transparent for the application, but is there any way it can inspect what is going on? I think about some kind of generalized getpeername(), which would allow the caller to get information about all subflows in use.
front a brief scan of 6824, the issue where a subflow finds a persistent error isn't clearly
that is we don't know if the frame has been received or not, and there doesn't seem to be any cross-subflow
ack machinery.
this is so obvious there is clearly an answer i didn't find...what is it?
For those wanting to try, their install howto. Seems supported on:
1. Linux - either debian binaries or compiling from source. Both kernel module and UserSpace ways.
2. Virtualized Linuxes - their example is provided for Amazon EC2
3. Mac OSX - but, obviously, not on iPhone (I estimate slim chances for this to happen in the near future - it's a technology disruptive for the mobile providers income, as it makes the multi-pathing over cell/WiFi hot-spots transparent to end user)
4. Android (Opinion: see? This is one of the reasons relying on "walled gardens" is bad: you have to wait for the mercy of the garden lord to benefit from something).
Questions raise, answers kill. Raise questions to stay alive.
Is anyone making network adapters with a built in programmable processor with an open specification these days? This particular protocol may well be compatible with some of the existing checksum offload implementations but GSO and GRO would seem to need something special especially for multiplexing across streams.
And the buffers get even bigger
So they're able to get 50Gbps out of 6 10Gbps circuits/handoffs/etc... well simple math would say why only 50 and not 60? Ok so it's cool if you want to tell me that I can simultaneously send data to a single destination via wifi and 3/4g, simultaneously being the key word. that part is interesting because in most cases the public address is different. But the article also mentions "servers that can pool multiple high speed interfaces". What? Any decent firewall, switch or server has been providing this functionality for years. It's called teaming or bonding. You want to send a shitload of data to a destination but it takes too long? Not a problem, throw a couple quad nics in those bitches and bond them up, problem solved providing your network can support the throughput. What am I missing?
This would be a dream for small businesses and home users. Have 2-4 DSL/Cable/Wireless WAN ports and one port for your LAN/Router. Plug and play for instant redundant internet at a much higher speed than any one low cost connection could provide.
or maybe we could just filter comments based on length or number of links. >1000 words or >20 links
Stop replying to trolls!
Akismet solved this problem 7 years ago. It's nuts that we are back to posters suggesting regex filters.
the internet really is a series of tubes.
Has anyone actually thought about how The Onipn Network might take advantage of this? This could potentially, or partially, fix one of Tor's biggest problems, which is the inherent low bandwidth due to one single TCP connection via several nodes in a serial manner. If Tor could leverage multiple nodes in parallel into a single connection, this ould certainly make it faster.... And even more anonymous tha it is today.
I was doing TCP multipath in 2004 using iptables to get more upstream out of my box at home. I had two 1.5/384 connections and could and up with 768k upstream. All it took was a clever iptables script that marked alternating packets - even and odd if you will - and mangled odd packets to go out one interface, and even packets for the other.
Obvioulsy, an actual TCP extension for this is going to be more elegant since it's more scalable and easy to deal with, but the idea is not new.
I find it annoying that my app is disrupted when I leave a Wifi and the (Android) phone needs a few seconds to connect to the #G/4G network. But then it might just to that because it needs to save the power on the network interface.
But then I'd buy the phone with double or tripple the battery over the slim one anyway. If only they would be available.
Who said anything about a regex?
Waiting to see an apartment full of geeks and gamers rocking 10 cable modems all linked together and splitting the bill.
NFS on multipath is my interest, too.
In NFS (v4) on TCP, the endpoints frame messages on the bytestream, independently in each direction. As FireFury03 states above, we're basically (potentially large) packet based. We'd like help from new transports in framing those messages optimally, avoiding head-of-line blocking for entire messages.
In addition to solving HOL, it's been proposed that we could design message framing on SCTP so as to deliver messages and data chunks on different streams, and get some advantages of NFS on RDMA.
I think it's the same story with a lot of protocols, including HTTP. In fact, like most web servers, the ONC RPC stack I work on is in user space, so I have a/the more complex version of these problems.
So do MTCP developers see solutions for any of these problems on the horizon?
Matt
So do MTCP developers see solutions for any of these problems on the horizon?
I'm in no way affiliated or knowledgeable in MPTCP - so I may be wrong - but from what I got from their presentation, MCTCP is not actually designed with the NFS-like usecase in mind (even if it may be used for such), but with the more "common" usecase of a mobile device able to use either/both cellular and WiFI networking.
The second thing that I saw as peculiar: it is not even supported by a network protocol (like IP is supporting TCP/UDP/SCTP/etc), but is supported by TCP. While it will have to deal somehow with re-assembling back a stream from packets streams over different paths, in itself it will be as prone to HOL as the bunch of underlying individual TCP streams which support those different paths.
That is, assuming the head-of-line is send on one path and that path involves a HOL-blocking, then it doesn't matter if the other paths have lower latencies, the entire original stream will be HOL-blocked
Questions raise, answers kill. Raise questions to stay alive.
That is, assuming the head-of-line is send on one path and that path involves a HOL-blocking, then it doesn't matter if the other paths have lower latencies, the entire original stream will be HOL-blocked
The implementation includes a solution to overcome HOL-blocking by reinjecting the blocking data-segment on the lower-latency path. Have a look at our scientific paper, which explains this mechanism: http://inl.info.ucl.ac.be/publications/how-hard-can-it-be-designing-and-implementing-deployable-multipath-tcp
this is awesome! too bad it had to take until 2013 for this to happen ; )
consider that WiFi has a good range of maybe 300 meters(?);
now imagine a road with multiple smallish houses each with 2 MBps ADSL, say like 10 of 'em.
if these house connect to each other with a wifi-bar-backbone and install this multiplath TCP,
then one house can use 20 MBps in the middle of the night : )
-
hope this makes it into the kernel and ALSO that the (available) GATEWAY(s) configuration is simplified:
http://multipath-tcp.org/pmwiki.php?n=Users.ConfigureRouting
is toooo difficult ; )
The implementation includes a solution to overcome HOL-blocking by reinjecting the blocking data-segment on the lower-latency path.
Have a look at our scientific paper, which explains this mechanism: http://inl.info.ucl.ac.be/publications/how-hard-can-it-be-designing-and-implementing-deployable-multipath-tcp
Oh, wow! Thanks.
Questions raise, answers kill. Raise questions to stay alive.
hey at ;east your thinking!
I used to be
Hi,
Speed-reading the paper, there appears to be some implicit ammunition for the SCTP approach (if it worked!), at least for applications like HTTP, NFS, etc, which are characterized by multiplexing of large and small messages on the stream. I conclude this from section 5.3, which I think states MPTCP over 2 links was slower than ordinary TCP over one link, when message size was 30K.
(Apologies if I'm misreading.)
Thanks,
Matt
Matt
I conclude this from section 5.3, which I think states MPTCP over 2 links was slower than ordinary TCP over one link, when message size was 30K.
For very small flow-sizes (like less than 30KB), MPTCP should not try to create additional subflows. Because, the whole data fits in the initial window of the first subflow. However, at the moment the linux implementation always tries to establish new subflows. In the paper's stress-testing scenario these additional subflows just consumed CPU-cycles and thus the "bad" results for MPTCP with very small flows
An easy fix would be to delay the establishment of additional subflows until a certain threshold of data has been sent or a certain time has passed.
Thanks, Christoph.
I think I was mis-reading, section 5.3 is discussing performance with short-lived connections, such as happens with HTTP .9 or 1.0. The question I would ask next is, how does MPTCP perform when HTTP 1.1 or similar channel multiplexing is used?
Matt
Matt
We have not yet done extensive experiments with HTTP1.1.
But, MPTCP is a benefit for bandwidth-intensive (thus increasing throughput) and long-lived connections (increased resilience against link-failures).
So, I would say that HTTP 1.1 would benefit from MPTCP.
Cheers,
Christoph
TCP X2 now with Shotgun technology!
Obviously with a protocol like SPDY/HTTP2 you have a much bigger chance reaping the benefits of MPTCP.
TCP is not tightly coupled with IP. You can put TCP on top of any network protocol you want. I have seen TCP on top of ATM, for example.