Slashdot Mirror
Better Networking with SCTP
5-0 writes to tell us that IBM DeveloperWorks has an interesting look at the key features of SCTP in the Linux 2.6 kernel and the ability to deliver multi-streaming. "SCTP is a reliable, general-purpose transport layer protocol for use on IP networks. While the protocol was originally designed for telephony signaling, SCTP provided an added bonus -- it solved some of the limitations of TCP while borrowing beneficial features of UDP. SCTP provides features for high availability, increased reliability, and improved security for socket initiation."
I don't know if you missed something - I didn't RTFA.
Heartbeats are optional. Some real-time applications probably want to use heartbeats every 10 seconds, while other can disable them completely.
The multihoming has nothing to do with routing table size. The multihoming feature is used for providing better connectivity.
Imagine your laptop with WiFi. If the application (say, FTP download) used SCTP instead of TCP then the download would not break when your laptop moves from one access point to another and switches ip-address. SCTP survives that.
Easy that: as long as it took IPv6 to be adopted into the mainstream.
Probably not that long. The problem with IPv6 is that too many entities are involved in a successful v6 deployment and too many changes have to happen at different levels.
OTOH, SCTP requires only a client and a server that want to use it.
Read about SYN Cookies. This is a method of avoiding SYN DOS attacks that has already been implemented in Linux (and probably elsewhere) for a while. I think SCTP just integrates the same concept into the official protocol specification. Once the SCTP server sends the INIT-ACK, it doesn't have to keep track of that association until the client sends a COOKIE-ECHO.
That's because IPv6 is *IP*. SCTP builds on top of IP (v4 if you want), just like UDP and TCP.
Just like many applications (mostly streaming servers and games, I suppose) use UDP without anybody caring, you can use SCTP without any host between client and server caring.
It's something for applications to use, not something that requires a different internet infrastructure, replacing routers, software etc. (IPv6 address syntax is different from the v4 one...).
Did I miss something?
This is an transport layer, not a network layer. It is only necessary in endpoints, such as clients and servers, and it might be a good thing if firewalls understood it. But the routers don't interpret it, so there won't be any change on backbones, except a slight increase in traffic with a few more keep-alive packets.
That's total bullshit. MS has nothing to do with this. In fact, no-support of SCTP in MS operating systems will be thee biggets hurdle for the introduction. It's *Linux* that is driving ther adoption !
Disclaimer: I'm using SCTP in my job for several years now.
Utterly incorrect.
If you had only taken ten seconds to check wikipedia's sctp page you would have found it was developed by the Internet Engineering Taskforce's SIGNTRAN working group.
The IETF is an open, all-volunteer standards organization and couldn't be further in spirit from the monopolies you mention.
Give credit where it is due indeed.
(oh and this protocol was defined in 2000 - far later then the telephone signalling I suspect you're thinking of)
My pics.
FYI,
p hp
http://www.altoriopreto.com.br/mestrado/index_en.
> Sctp is new
Nope. I worked on SCTP implementation in year 2000.... Nortel had it in 1999.
zm
Sig ?
Seems like there is an implementation for FreeBSD and Darwin (underlying OS used by MacOS X) too, according to this page.
Jumpstart the tartan drive.
There's also Scalable-TCP, High-Speed TCP, FAST-TCP, BIC-TCP, H-TCP. Each with their own advantages. Check out the site. These guys are doing interesting evaluations. H-TCP is specifically what they work on:
TCP Evaluation Discussion
Interesting plots too
The end result is that TCP is not particularly suited to high-speed networks.
In fact, that's pretty close to how it's done according to SCTP for beginners
I may run Linux myself, but in almost everything I do on my desktop (that isn't itself Linux-related) I am interacting with non-Linux machines. I'm forever "losing out" because I can't receive MSN special features. Sure I could do webcam with what was gnomemeeting (it looks awesome) but does anyone run it? Thankfully now I have friends riding Firefox and one using Jabber (googletalk).
But yes all my friends use windows!
So will such features help Desktop Linux?
Short answer: It might "help Desktop Linux" in general, but it will fix zero interoperability issues and it will do nothing to the problems you listed.
Long answer: You need to learn a few things about network protocols, my friend. Even if SCTP, TCP or UDP had anything to do with your problems, SCTP is not implemented on Windows. Most if not all of the programs you're using use TCP or UDP, and the issues of compatibility you're experiencing have nothing to do with these protocols. The programs you mention have their own protocols that run over TCP and UDP. Seriously, go and learn how to program BSD sockets and you'll understand where TCP and UDP are in the network protocol heirarchy. Once you've done that, maybe you could help out projects like Kopete and Gaim to fix your problems.
I have discovered a truly remarkable proof of this theorem that this sig is too small to contain.
The SYN flood happens because the server has to keep track of the connection until it times out and there's a limit to the number of connections the server can keep track of, with the cookie method the server gets an INIT, sends out the cookie and forgets about the connection. Only when that is returned the server opens a connection. Sure, you could go through the proper protocol for starting the connection but that forces you to tell the server the real IP of your DoSing client instead of putting a number of fake IPs in there. The server could just start ignoring your client for a while and your DoS has failed.
Justice is the sheep getting arrested while an impartial judge declares the vote void.
DCCP and SCTP are not very related. DCCP is an improvement on UDP - DCCP is an unreliable protocol that improves upon UDP by being aware of, and throttling back upon, network congestion.
For all its problems TCP/IP is everywhere. This fact has made it the networking technology to use even when it doesn't make technical sense. For example, folks use it in high performance computing and in storage (iSCSI) where there are much better methods available technically. Its commonality (along with ethernet's popularity) often make TCP/IP over ethernet the cheapest solution to many problems (while not the best).
I used to work on InfiniBand where the reliability/congestion detection protocol (Reliable Connected and Link Level flow control in IB terms) are in hardware. This scales to 20 Gbit connections between hosts quite well. Other examples of hardware protocols include myrinet (invented by myricom) and qsnet (from quadrics) and scalable coherent interface current pushed by Dolphin Interconnect. All of these folks struggle to compete with good old TCP/IP over ethernet. Except for the parts of the HPC world, TCP/IP over ethernet wins. In the storage landscape, Fibre Channel, SAS, and SATA seem to be holding out but iSCSI sure is trying.
The performance issue is real though and very few systems can saturate a 10 Gbit TCP/IP etnernet link without massive host CPU overhead. One solution floating around is that instead of trying to make new protocols to replace TCP, we should imitate the competition and put hard work in hardware. TCP/IP offload NICs (TOE) are becoming increasingly more popular. With RDMA technology layered on top of it you get iWARP. For storage you get iSER (ironically from an IB company!). This technology is being adopted by both the MS and Linux camps so it seems to have a good shot. In fact, many of the interfaces used by IB work about as well over iWARP cards. Things like Message Passing Interface, Direct Access Provider Library, Sockets Direct Protocol (SDP), and iSER do not know the difference between iWARP and IB or anyone else.
Software can just post a full size message and it gets sent out the wire without copying, segmentation, timers, resends, or other CPU hogs. This kind of stuff really helps with large messages. With SDP, apps can be made to take advantage of it without changes to the application. MS is also providing a standard way for just TOE NICs without RDMA abilities to work with the OS. Linux doesn't seem to have a standardized way for TCP/IP to be offloaded entirely but is supporting RDMA and SDP.
The things SCTP seems to offer is more explicit understanding of the difference between failure and congestion and multi-home support. This could make load balancing over multiple paths between hosts pretty interesting. The problem I see is that is that it is competing with the established TCP that now has many of its warts fixed with hardware offload. SCTP will still have the issue of a CPU handling segmentation/reassembly, massive amounts of interrupts, timer/retry overhead, etc. It also seems to have a higher overhead for connection establishment (although that is mitigated by being able to send data during the end phases). Is this a solution looking for a real problem? Pehaps not. Does this really have a chance of being taken up? I am not too confident.
-Ack
-- soldack
The first attempt in the IP world to add a protocol of this type was Reliable Data Protocol, in 1984. (See RFC 908). But that never went anywhere. Since then, nobody has really addressed this. There was ISO TP4, but that didn't go anywhere either, althoug it was fully supported in Windows NT.
SCTP has reasonable congestion behavior, like TCP, so it's an improvement over UDP-based protocols in that regard. Moving some UDP-based protocols to SCTP could be a step up. That's where it could be most useful. It might make sense to put remote procedure call type protocols that now use UDP onto SCTP. If a protocol has to do retransmission, it's better to do it at the transport layer than at the application layer.
The "multihoming" thing seems badly placed, because that's not properly a transport layer function. But I haven't really looked at that.
John Nagle
SCTP does have an option for using name resolution to do multihoming, however for practical reasons it is almost universally unimplemented. SCTP multihoming works just fine without it. IP address lists for multihoming are exchanged during the standard connection (association) establishment process.
State cookies are not stored on the server at all, but rather are echoed from the client back to the server as a effective means of SYN flood style DoS attack prevention.
SCTP (properly implemented) is radically superior to TCP for a large class of applications, basically anything that needs low latency reliable message exchange. The lack of message boundary information in TCP causes considerable pain for implementers of upper layer protocols - notably RDMA/RDDP and iSCSI. The running solution for efficient hardware implementation of RDMA and iSCSI over TCP involves *inserting* markers every 512 bytes or so in the middle of a data stream so that the receiver can re-synchronize it efficiently.
The primary SCTP RFC is RFC 2960 for those who are wondering.
You, Sir, are a blatant troll. I happen to know at least one of the developers of SCTP (was one of my professor at university), and Microsoft has nothing to do with this. Stop spreading bullshit.
This sounds somewhat similar to TIPC which we're using in some projects where I work. Like UDP it is message based, but it provides a reliable message transport. It also runs in the kernel as a protocol stack. It does have some differences, though. It is not based on a source and destination, but rather a publish/subscribe mechanism which sounds similar to the SCTP multi-homing support. With the publish/subscribe, one or more clients can indicate that they're interested in a certain service. When that service becomes available or disappears on the node, cluster, or network (depending on the scope of configuration) the client stack will automatically notify it.
It also has the concept of priority in it, so that messages may be prioritized.
Unlike SCTP, however, it does not run on top of IP but is its own protocol that runs directly over the wire, which means that it cannot be routed across an IP network. It is great as an internal embedded messaging protocol, but not as useful when a network is involved.
TIPC is also not connection oriented. There is no connection setup required to send messages much like UDP.
-Aaron
This post is encrypted twice with ROT-13. Documenting or attempting to crack this encryption is illegal.
SCTP excels at low latency delivery of small messages. TCP's head of line blocking is a serious problem in many applications. SSH tunneling is a good example.
The main advantage of using SCTP over multiple TCP connections is connection establishment time as well as server overhead. You can create an association with hundreds of streams in the roughly the same time as a single TCP connection, with little or no overhead for unused streams. Then when you want to initiate a new non-blocking transaction you can send a message on a new stream without the three-way handshake of an extra TCP connection.
In addition, a single SCTP socket can handle reliably delivered messages on thousands of streams from hundreds of associations. No need to use select()/poll() on a long list of file descriptors.
SCTP and Infiniband focus on different areas. IB is largely a high performance HPC / cluster network architecture for LAN applications, where SCTP is a transport protocol designed to operate efficiently under WAN conditions (significant packet loss, high RTTs).
SCTP is a more efficient RDMA/iWARP transport than TCP, but the differential advantage of SCTP over TCP is much lower in a LAN environment due to the low RTTs, so RDDP/TCP dominates so far despite the bizarre marker insertion scheme (MPA). Same goes for iSCSI.
The interrupt issue has largely been solved - on Linux NAPI dynamically switches between interrupt and polled mode to reduce this overhead to negligible levels. Message signalled interrupts also help considerably.
What would be much more helpful (and economical) for iSCSI, SCTP, and RDDP is NIC CRC32C checksum generation. CRC generation is quite expensive in software but trivial in hardware.
SCTP wasn't originally designed for load balancing a single association via simultaneous multi-path transfer (SMT). It can be done, but it requires some loss detection algorithm changes. Someone still needs to develop a option to coordinate this at association establishment time.
One advantage of SCTP over TCP is that on a per stream basis, SCTP connection establishment overhead is much lower than TCP - basically O(1) instead of O(N) in the number of streams.
Congestion control is an area where SCTP is much like TCP. SCTP uses AIMD on a per destination address basis. However any of the the alternative congestion algorithms for TCP would behave similarly with SCTP.
Of course given the additional message boundary information available in SCTP, further improvements could be made.
Some of the protocols that could benefit from SCTP include:
now we need to go OSS in diesel cars
Actually, most iWARP/RDMA stuff doesn't have a software interface to TCP at all - the hardware handles not only TCP, but three or more layers on top of it (at least MPA, DDP, and RDMA, plus iSCSI in some cases). That type of interface is not a problem. What is controversial is using TOE for conventional TCP applications using kernel space dispatch.
This is a bit of an end run around the Linux kernel bridging, routing, and filtering layers, which is the primary reason why support for it won't get merged in the kernel socket layer until RNICs can at a minimum do IPtables like IP address filtering and proper dispatch so that some packets can be routed through the kernel layers on an Ethertype and IP protocol / address / port specific basis.