Slashdot Mirror
Better Networking with SCTP
5-0 writes to tell us that IBM DeveloperWorks has an interesting look at the key features of SCTP in the Linux 2.6 kernel and the ability to deliver multi-streaming. "SCTP is a reliable, general-purpose transport layer protocol for use on IP networks. While the protocol was originally designed for telephony signaling, SCTP provided an added bonus -- it solved some of the limitations of TCP while borrowing beneficial features of UDP. SCTP provides features for high availability, increased reliability, and improved security for socket initiation."
How long do you think it will be before this is adopted into the mainstream?
Easy that: as long as it took IPv6 to be adopted into the mainstream.
"A door is what a dog is perpetually on the wrong side of" - Ogden Nash
The article makes SCTP sound like it's the greatest thing since sliced bread. It's as fast as UDP, reliable as TCP, and is not susceptible to SYN floods like TCP. It's amazing! It's the fastest, it's the quickest, it's the best!
Really?
I don't know if you missed something - I didn't RTFA.
Heartbeats are optional. Some real-time applications probably want to use heartbeats every 10 seconds, while other can disable them completely.
The multihoming has nothing to do with routing table size. The multihoming feature is used for providing better connectivity.
Imagine your laptop with WiFi. If the application (say, FTP download) used SCTP instead of TCP then the download would not break when your laptop moves from one access point to another and switches ip-address. SCTP survives that.
Second City Transport Protocol!? John Candy would be proud!
Easy that: as long as it took IPv6 to be adopted into the mainstream.
Probably not that long. The problem with IPv6 is that too many entities are involved in a successful v6 deployment and too many changes have to happen at different levels.
OTOH, SCTP requires only a client and a server that want to use it.
Wrong. A connection with a forged source address won't take any more resources than a single incoming packet, a single outgoing packet and the CPU cost of computing a cookie. That's all.
Flooding using the flooder's true address will still work, but it is trivial to block. Sure, having 100000 zombies flood a single destination will put quite a burden and will force the floodee to maintain a huge list of banned addresses, but, a single hash table on the router will alleviate anything except for bandwidth wasted.
This is same as a full TCP connect() flood.
There is a TCP hack named "syn cookies", but this doesn't work very well as TCP wasn't designed to be resistant to SYN floods.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
Read about SYN Cookies. This is a method of avoiding SYN DOS attacks that has already been implemented in Linux (and probably elsewhere) for a while. I think SCTP just integrates the same concept into the official protocol specification. Once the SCTP server sends the INIT-ACK, it doesn't have to keep track of that association until the client sends a COOKIE-ECHO.
OTOH, SCTP requires only a client and a server that want to use it.
And no overzealous firewalls on the way.
The creatures outside looked from Alt-Right to Antifa; but already it was impossible to say which was which.
That's because IPv6 is *IP*. SCTP builds on top of IP (v4 if you want), just like UDP and TCP.
Just like many applications (mostly streaming servers and games, I suppose) use UDP without anybody caring, you can use SCTP without any host between client and server caring.
It's something for applications to use, not something that requires a different internet infrastructure, replacing routers, software etc. (IPv6 address syntax is different from the v4 one...).
lets change the quote scope a little:
SCTP protects against this type of attack through a four-way handshake and the introduction of a cookie. In SCTP, a client initiates a connection with an INIT packet. The server responds with an INIT-ACK, which includes the cookie (a unique context identifying this proposed connection). The client then responds with a COOKIE-ECHO, which contains the cookie sent by the server. At this point, the server allocates the resource for the connection and acknowledges this by sending a COOKIE-ACK to the client.
Funny how things suddenly makes sense when you read the entire paragraph.
Did I miss something?
This is an transport layer, not a network layer. It is only necessary in endpoints, such as clients and servers, and it might be a good thing if firewalls understood it. But the routers don't interpret it, so there won't be any change on backbones, except a slight increase in traffic with a few more keep-alive packets.
That's total bullshit. MS has nothing to do with this. In fact, no-support of SCTP in MS operating systems will be thee biggets hurdle for the introduction. It's *Linux* that is driving ther adoption !
Disclaimer: I'm using SCTP in my job for several years now.
FYI,
p hp
http://www.altoriopreto.com.br/mestrado/index_en.
How long do you think it will be before this is adopted into the mainstream?
Probably never main stream. Maybe for some telco types in niche applications... but it is too easy for 99% of the world to just open 2 sockets if you want 2 streams, or rpc's and threads... both of which are well supported and seasoned. Sctp is new, new bugs, not supported everywhere and as a result will go not go far.
One might argue it is supposed to be more secure, I argue it is not. If it was it would be tied to kerberos and ipsec and use AES at the transport layer.
Sctp has only one advantage, and this too could be done using TCP or UDP with not too much effort. That is you can open one socket and have mutiple streams inside, reducing the socket count on servers, a problem if you are routing more than 48000 calls or so. But yor could also do this with "TCP connecting pooling", a common way around this issue.
But like ATM, it is the telco business push. ATM anyone?
Sctp to me looks like a problem looking for a home for 99% of us. But at least an informative post so when I see the compile option I will turn it off.
Seems like there is an implementation for FreeBSD and Darwin (underlying OS used by MacOS X) too, according to this page.
Jumpstart the tartan drive.
A while ago I read the RFC. It is very scary. Multihoming as proposed moves things like name resolution into the kernel.
I will grant SCTP does some neet stuff, the best is that it allows independent non-mutually-blocking streams over one connection. It also has state cookies, yum.
SCTP tries to be all things to all people in one protocol. It reads as though they just decided the whole layered protocol thing was overrated and shoved every new feature into this one layer.
The Linux network stack is having tons of new things lately, SCTP is one, but how it compares with DCCP, which has also been implemented and merged in Linux?
The wikipedia assumes they share some properties, but it's SCTP a better DCCP, or what?
There's also Scalable-TCP, High-Speed TCP, FAST-TCP, BIC-TCP, H-TCP. Each with their own advantages. Check out the site. These guys are doing interesting evaluations. H-TCP is specifically what they work on:
TCP Evaluation Discussion
Interesting plots too
The end result is that TCP is not particularly suited to high-speed networks.
In fact, that's pretty close to how it's done according to SCTP for beginners
I may run Linux myself, but in almost everything I do on my desktop (that isn't itself Linux-related) I am interacting with non-Linux machines. I'm forever "losing out" because I can't receive MSN special features. Sure I could do webcam with what was gnomemeeting (it looks awesome) but does anyone run it? Thankfully now I have friends riding Firefox and one using Jabber (googletalk).
But yes all my friends use windows!
So will such features help Desktop Linux?
Short answer: It might "help Desktop Linux" in general, but it will fix zero interoperability issues and it will do nothing to the problems you listed.
Long answer: You need to learn a few things about network protocols, my friend. Even if SCTP, TCP or UDP had anything to do with your problems, SCTP is not implemented on Windows. Most if not all of the programs you're using use TCP or UDP, and the issues of compatibility you're experiencing have nothing to do with these protocols. The programs you mention have their own protocols that run over TCP and UDP. Seriously, go and learn how to program BSD sockets and you'll understand where TCP and UDP are in the network protocol heirarchy. Once you've done that, maybe you could help out projects like Kopete and Gaim to fix your problems.
I have discovered a truly remarkable proof of this theorem that this sig is too small to contain.
Nope. I worked on SCTP implementation in year 2000.... Nortel had it in 1999.
New as in it is just making it into some kernels. And it most of us have never seen an application use it. And it may be years before we do. However, as stated it exists in a niche telco market.
Nortel (used to work there) and the industry still has the "central office" mentality. Nortel had one thing right, VoIP is the future. What the telco business as a whole has wrong is how this will be done.
In the future there will no need for a central office, all calls will NOT route through central servers and thus negate a heavy need for sctp altogether! sctp is like a T1-T2-Txxx to sockets, allowing n channels of calls through one IP connection. If VoIP (not strictly defined) goes point to point direct there is no need for a central office. End user devices only need 1 to 4 channels. (Audio/Video/Control/MP3 Movies).
What will happen is someone like Linksys (or a Chinese company) will get enough momentum to produce a $99 device you hook to your internet, some LDAP server out there will be your directory and call routing will go direct device to device over TCP/IP. The MOBILE protocol has a better chance of surpasing SCTP as being in common use. You might even run call conferencing right off your own device.
Central office technology has seen it's peek hayday. SS7, BSSMAP, ISDN, SONET and others are far too complex, expensive, patented and cumbersome - and will be religated to legacy wireline only. SCTP might be used in this niche area to concentrator like a RLCM to wireline services. Hardly end user equipment.
The Internet is slowly eatting the telco business alive. As the traditional telco business market is evaporating.
Wireline needs to quite the bickering, quite tripping on DCLEC cables and get decent inexpensive DSL services or they can say good-bye to their business. DSL is the only hope for the wireline side of the telco business and most are screwing it up big time.
Cisco, if they keep innovation going high are going to put Nortel out of business. Central offices are being replaced with Network Access Point (NAP) and Cisco is king. Nortel might be best to spend their efforts on making the biggest, fastest, cheapest internet router possible. A DMS10000, 10000 as it can take 10000 IP based fibers.
BTW, I loved working for Nortel, but left as I was a grossly underpaid Canadian and could see Stern was going to wreck the company.
For all its problems TCP/IP is everywhere. This fact has made it the networking technology to use even when it doesn't make technical sense. For example, folks use it in high performance computing and in storage (iSCSI) where there are much better methods available technically. Its commonality (along with ethernet's popularity) often make TCP/IP over ethernet the cheapest solution to many problems (while not the best).
I used to work on InfiniBand where the reliability/congestion detection protocol (Reliable Connected and Link Level flow control in IB terms) are in hardware. This scales to 20 Gbit connections between hosts quite well. Other examples of hardware protocols include myrinet (invented by myricom) and qsnet (from quadrics) and scalable coherent interface current pushed by Dolphin Interconnect. All of these folks struggle to compete with good old TCP/IP over ethernet. Except for the parts of the HPC world, TCP/IP over ethernet wins. In the storage landscape, Fibre Channel, SAS, and SATA seem to be holding out but iSCSI sure is trying.
The performance issue is real though and very few systems can saturate a 10 Gbit TCP/IP etnernet link without massive host CPU overhead. One solution floating around is that instead of trying to make new protocols to replace TCP, we should imitate the competition and put hard work in hardware. TCP/IP offload NICs (TOE) are becoming increasingly more popular. With RDMA technology layered on top of it you get iWARP. For storage you get iSER (ironically from an IB company!). This technology is being adopted by both the MS and Linux camps so it seems to have a good shot. In fact, many of the interfaces used by IB work about as well over iWARP cards. Things like Message Passing Interface, Direct Access Provider Library, Sockets Direct Protocol (SDP), and iSER do not know the difference between iWARP and IB or anyone else.
Software can just post a full size message and it gets sent out the wire without copying, segmentation, timers, resends, or other CPU hogs. This kind of stuff really helps with large messages. With SDP, apps can be made to take advantage of it without changes to the application. MS is also providing a standard way for just TOE NICs without RDMA abilities to work with the OS. Linux doesn't seem to have a standardized way for TCP/IP to be offloaded entirely but is supporting RDMA and SDP.
The things SCTP seems to offer is more explicit understanding of the difference between failure and congestion and multi-home support. This could make load balancing over multiple paths between hosts pretty interesting. The problem I see is that is that it is competing with the established TCP that now has many of its warts fixed with hardware offload. SCTP will still have the issue of a CPU handling segmentation/reassembly, massive amounts of interrupts, timer/retry overhead, etc. It also seems to have a higher overhead for connection establishment (although that is mitigated by being able to send data during the end phases). Is this a solution looking for a real problem? Pehaps not. Does this really have a chance of being taken up? I am not too confident.
-Ack
-- soldack
The first attempt in the IP world to add a protocol of this type was Reliable Data Protocol, in 1984. (See RFC 908). But that never went anywhere. Since then, nobody has really addressed this. There was ISO TP4, but that didn't go anywhere either, althoug it was fully supported in Windows NT.
SCTP has reasonable congestion behavior, like TCP, so it's an improvement over UDP-based protocols in that regard. Moving some UDP-based protocols to SCTP could be a step up. That's where it could be most useful. It might make sense to put remote procedure call type protocols that now use UDP onto SCTP. If a protocol has to do retransmission, it's better to do it at the transport layer than at the application layer.
The "multihoming" thing seems badly placed, because that's not properly a transport layer function. But I haven't really looked at that.
John Nagle
SCTP does have an option for using name resolution to do multihoming, however for practical reasons it is almost universally unimplemented. SCTP multihoming works just fine without it. IP address lists for multihoming are exchanged during the standard connection (association) establishment process.
State cookies are not stored on the server at all, but rather are echoed from the client back to the server as a effective means of SYN flood style DoS attack prevention.
SCTP (properly implemented) is radically superior to TCP for a large class of applications, basically anything that needs low latency reliable message exchange. The lack of message boundary information in TCP causes considerable pain for implementers of upper layer protocols - notably RDMA/RDDP and iSCSI. The running solution for efficient hardware implementation of RDMA and iSCSI over TCP involves *inserting* markers every 512 bytes or so in the middle of a data stream so that the receiver can re-synchronize it efficiently.
The primary SCTP RFC is RFC 2960 for those who are wondering.
SCTP and Infiniband focus on different areas. IB is largely a high performance HPC / cluster network architecture for LAN applications, where SCTP is a transport protocol designed to operate efficiently under WAN conditions (significant packet loss, high RTTs).
SCTP is a more efficient RDMA/iWARP transport than TCP, but the differential advantage of SCTP over TCP is much lower in a LAN environment due to the low RTTs, so RDDP/TCP dominates so far despite the bizarre marker insertion scheme (MPA). Same goes for iSCSI.
The interrupt issue has largely been solved - on Linux NAPI dynamically switches between interrupt and polled mode to reduce this overhead to negligible levels. Message signalled interrupts also help considerably.
What would be much more helpful (and economical) for iSCSI, SCTP, and RDDP is NIC CRC32C checksum generation. CRC generation is quite expensive in software but trivial in hardware.
SCTP wasn't originally designed for load balancing a single association via simultaneous multi-path transfer (SMT). It can be done, but it requires some loss detection algorithm changes. Someone still needs to develop a option to coordinate this at association establishment time.
One advantage of SCTP over TCP is that on a per stream basis, SCTP connection establishment overhead is much lower than TCP - basically O(1) instead of O(N) in the number of streams.
Some of the protocols that could benefit from SCTP include:
now we need to go OSS in diesel cars
Actually, most iWARP/RDMA stuff doesn't have a software interface to TCP at all - the hardware handles not only TCP, but three or more layers on top of it (at least MPA, DDP, and RDMA, plus iSCSI in some cases). That type of interface is not a problem. What is controversial is using TOE for conventional TCP applications using kernel space dispatch.
This is a bit of an end run around the Linux kernel bridging, routing, and filtering layers, which is the primary reason why support for it won't get merged in the kernel socket layer until RNICs can at a minimum do IPtables like IP address filtering and proper dispatch so that some packets can be routed through the kernel layers on an Ethertype and IP protocol / address / port specific basis.
SCTP sounds like a kitchen sink solution; it has some nice features and some useless features.
What's useless to one application is useful to another. Most of the features can be turned on and off, so the application developer can pick what's suitable for their use.
For example, manually opening multiple connections through different interfaces and then having the SCTP implementation figure out which one to send through is nonsense; if the system has multiple routes to the Internet, then that can be taken care of at the IP level.
This is one thing that I almost agree with you on - multihoming should probably be done at the IP level. But that requires that intermediate routers be modified to introduce the required functionality and we have already seen that many ISPs have no interest in adjusting their infrastructure to support new technologies (multicast, IPv6, etc). SCTP's multihoming support has the advantage that only the end points of the connection need to care, to the rest of the network it's just plain old IPv4.
Similarly, preservation of write boundaries is a useless gimmick that is rarely needed, and when it is needed, can be easily implemented in user code.
I'm not sure why you think this is a "useless gimmick". Very few applications want a byte stream - almost everything works on the datagram level. Think about HTTP - you send the server a bunch of headers (these are separate datagrams), the server returns a bunch of headers (again, separate datagrams) and the actual object data (one massive datagram). At the moment this is done over a byte stream and in order to maintain the boundaries between the datagrams you have to delimit them at the sending end and then parse them at the receiving end. With almost every application wanting to send multiple datagrams instead of a byte stream isn't it better to have this handled at the protocol level rather than reimplementing it for every application? Almost the only things which benefit from byte streams rather than datagram streams are interactive stuff like telnet and SSH (even SSH would benefit from SCTP when you're multiplexing multiple tunnels)
The four-way handshake during setup is possibly useful, but you can trivially get the same with TCP in a backwards compatible fashion if you configure your kernel to protect against SYN spoofing.
TCP SYN cookies are weak in comparison to the SCTP 4-way handshake.
Altogether, I'm not quite sure what problem SCTP is supposed to solve. SCTP has made its way into some other standards, so it will probably be unavoidable, but it's not a well-designed protocol in my opinion.
SCTP was originally designed for telephony applications (it is used to transport SIGTRAN traffic and can also be used to transport SIP). It is designed to combine the benefits of TCP (reliable ordered delivery with congestion control) with the benefits of UDP (preservation of message boundaries and unordered delivery). But while designing a new protocol it was worthwhile addressing other problems that have shown up with TCP and UDP. I would hazard a guess that the _only_ reason TCP is so widely used is because it's the only widely available transport that provides congestion control and reliable ordered delivery - most applications are _not_ suited to communicating through byte streams and many do not even require the data to arrive in order. If SCTP is widely available as an alternative protocol I can see it being used for new applications purely because the preservation of message boundaries removes the need for a chunk of parsing code.
http://blog.nexusuk.org