I wouldn't argue in favor of serving static, linearly dependent and linearly organized web pages with a UDP-based custom layer. I wouldn't argue against it, but it's not one of those cases where you just slap your forehead at the stuff TCP does to get in your way.
There's a good chance that if you set out to design a transport layer for HTTP you'd end up reimplementing TCP badly, as others in the thread have pointed out. But it's not 1988 anymore, and modern networked applications do a lot of stuff that they didn't do back then. There's no need to reinvent the wheel, but there's also no need to put wheels on a jetpack.
All running applications using the link together know better than TCP/IP how to allocate bandwidth but no one application does.
Gee. <scratches head> If only there were a way for multiple clients to optimize their traffic patterns on a cooperative basis. Something like, perhaps, a network of some sort.
I'm obviously not going to write a treatise on RFC 2581 in a Slashdot comment.:-P Suffice it to say that the decision to employ most of those methods -- slow start, congestion avoidance, fast retransmit and fast recovery -- belongs in the app because it simply cannot be optimally implemented by a content-agnostic layer. (If your network depends on well-defined slow start behavior... well, that's a shame, isn't it. Light up some more fiber and send Cisco some more bucks.)
And yes, most of that RFC discusses how, when, and how often to naively retransmit data that the application probably doesn't care about anymore.
So, phrasing this as something other than a flame, you would suggest that HTTP, FTP, etc. use UDP and then let each client application implement its own rate limiting
Because my application is better at knowing how to adapt to a given amount of bandwidth than TCP/IP is.
retransmission policy
Because non-trivial applications often have better ways to deal with packet loss than TCP's acknowledgement/retransmission mechanism....and packet reordering?
It often makes more sense to let your application deal with this. BitTorrent's a great example, as are most action games that do client-side prediction. Likewise, the way streaming applications have to work in the presence of TCP's reordering is by grinding to a halt waiting for the retransmission. Gee, thanks, TCP. Instead of a 16-millisecond glitch, I get a 2000-millisecond dropout.
Or did you have some other exotic transport layer in mind to replace TCP
No; I believe that dictating those details to application developers is counterproductive. As long as the implementation is competent, datagram transport with application-specific reliable delivery is almost always a win for everybody concerned -- clients, servers, routers. And if the implementation is incompetent, TCP can only do so much to help.
What? Why? Why is a connection based application "bad design"?
Because connection maintenance is a very high-level task that should not be delegated to the very bowels of the application, much less to someone else's opaque API.
A "session" is just a hack to give you connection-like qualities over a connectionless protocol, such as HTTP. If you want connection-like behavior, and you're free to design your own protocol, why not just use connections? Why make your life more complicated?
The complication comes in when you have to code "connected versus unconnected" states into your app all the way up to the user-interface level, and you realize that you're doing a lot of redundant work to try to keep track of what TCP/IP thinks the connection state is. Admittedly, it's hard to explain unless you've been there.
There are plenty of advantages to connection-based applications; no need to re-authenticate on every message, for one.
TCP is doing this behind your back. Maybe you can do it better.
Clear indications of when a client is done talking to you for another (the connection is dropped, whereas in a connectionless world you never really know if the client has died or is just about to send you another message).
That's one of the worst examples you could pick. When, and whether, TCP will drop a connection is up to factors far beyond the application's control. You have to handle the "TCP dropped connection" state anyway; it's not really any easier to do that at the lowest networking level than it is to do it at a higher level.
Most applications I've worked on have ended up with their own keepalive-based housekeeping mechanisms anyway; the obvious tactic is to use those mechanisms for connection maintenance instead of a low-level TCP connection whose persistence your own code has to mirror.
Even most "connectionless" applications, such as your web browser, try to cheat and get connection based behavior.
Again you're making my point for me. Because TCP-level connections are not appropriate for Web browsing, various Procrustean hacks get brought into play.
Renegotiating an SSL connection for every request would be remarkably expensive for the server, so the server and browser try to reuse the same connection as much as possible. Fire up wireshark, and you'll see your browser will only open two connections when you read Slashdot, and all the images and text all come across those two connections.
Sure, but that has nothing to do with any inherent advantages of TCP-level connections, except insofar as SSL is tied to them. If you were driven by absolute maximization of server performance and weren't saddled with the baggage of existing protocols like HTTP, you certainly wouldn't use TCP; it'd be like doing unnecessary work in the inner loop of a graphics routine. That's the decision the BitTorrent people are faced with.
TCP does work well, and certainly the axiom about those who don't understand TCP being condemned to reimplement it badly is a valid one. However, it is the wrong tool for many modern applications. If you're writing networking code and you find yourself fighting TCP's behavior, it's not necessarily your fault.
The developers of BitTorrent have long since passed the level of play where they're better off using TCP/IP because "well, because that's what you're supposed to use." My objection was to the naive canonization of TCP as an all-purpose protocol that's somehow magically superior to any protocol that the application can implement on its own behalf.
"Because it's the most router-friendly protocol" is also no excuse. Applications that aren't a good fit for TCP often abuse the protocol, e.g. by disabling Nagling and tinkering with other parameters in an effort to make it work more like, well, UDP.
Those developers didn't reimplement TCP badly, but they might as well have, as their efforts are likely to have the same effect.
To say that TCP is optimal at dealing with congestion is to say that individual packets are always a good representation of the data blocks being sent and received at the application level, and that best thing that any application can do when expected data doesn't arrive is to wait on it to be retransmitted, with the network layer queuing up all subsequent intact packets.
Once again, this behavior is guaranteed to be completely wrong for anything but toy command-line applications that fit on a single page in the back of a musty-smelling manual.
There's no way that BT's protocol could be as sophisticated as TCP, given its 30+ years of development. Most people don't appreciate how amazingly well TCP's flow control works in terms of maximizing link utilization in a way that is fair to all network users. We really don't need is an arms race of new, greedier protocols.
TCP gets a lot of credit it doesn't deserve. It enforces bad design -- most client/server applications should be either stateless or session-based, rather than connection-oriented. Anything that even vaguely resembles a streaming application shouldn't even consider TCP. Finally, TCP's connection model is almost guaranteed to be suboptimal for any application that does require one.
What are the odds that HTTP, FTP, SMTP, and BitTorrent will all work optimally over TCP? Slim to none, and none is still waiting for Nagle.
I'm not an libertarian in any way, but prohibiting things just because they are dangerous to the same person that chooses to do those things is stupid and harmful to everybody.
That's more or less the definition of "libertarian." If someone isn't either harming you directly or about to do so, let them be.
I didn't exactly respect him more after reading Travels. It was hard to escape the conclusion that Crichton was a guy who would believe literally anything anyone told him. That's one reason I was somewhat surprised to see him arguing in favor of more objective thinking in the global-warming debate. His bio made him sound like a real New Age woo-woo type.
Make no mistake, the american dream is the ultimate enslavement tool. Because the truth of course is that NOT everyone can make it, no matter how they try because a capatalist system needs its homeless to allow for the superrich. The american dream at best is an lottery, but one where the winners can buy the winning tickets.
Funny, though, everybody who tries to improve on this system ends up killing tens of millions of their own people. Any suggestions for avoiding that?
<shrug> I'm not a political guy, really. Just wanted to point out to the original poster that the time to get all panicky about "creeping socialism" was 70 years ago. It is way too late now, for better or for worse. These days, the debate on socialistic government can only address "how much," not "if" or "when."
Congratulations in reading a value judgment into my post that wasn't there. Now go call your doctor -- private, socialized, or otherwise -- about that twitching knee.
Lets start at the beginning. Grandparents on mother side were both communists. Mom was at least an anti-american, probably socialist leaning if not a fellow traveller. Dad? Communist. What else do you call a man who gets tossed out of an overtly Socialist government for being to much of a redistributionist? Growing up Grampa figures Young Obama needs a proper father figure who can teach him how to be black enough to properly hate on whitey so he fixes him up with his old CPUSA buddy Frank Marshal Davis. Poor kid never really stood a chance, but then EVERYBODY at some point has to decide whether to keep the morals they grew up with or not.
What will really bake your noodle is if, like me, you ever run across an old socialist propaganda tract from the 1930s era in a used-book store. You will feel like you took the red pill with an LSD chaser; you will see that the agitators among Obama's ancestors won out in the end. They have gotten 98% of everything they ever asked for, from Social Security, to expansion of Federal oversight over the banking and financial sectors, to legal recognition for organized labor, to racial and sexual parity.
It's stunning when you realize that we are not only staring down a socialist path as a nation, but we've been on one for the last 80 years. There truly is very little new under the Sun. Obama likely represents nothing more than the status quo... which ironically makes him the real Conservative.
The Republicans running these days are a bunch of nation-building Jesus freaks who have long abandoned any claim to conservative ideology. (I mean, really. How else can a Republican define "epic fail" than to be kicked out of government in favor of an attractive and successful African American named Hussein?!)
I say let's elect someone from the right side of the bell curve for a change, just to see what happens.
I can never understand the cadre of poor and middle class who feel the need to defend the rich. Do you think defending the rich will make you rich? Do you think the rich will accept you as anything but a quaint and loyal pet? Or maybe you feel the tax burden, and are just pissed off about taxes? If that's the case, don't you realize that you, the poor or middle class, are paying an unfair burden at the expense of the wealthy?
Nor can I ever understand people who divide other people into Manichaean classes like you're doing. Economic policies are either sound or unsound, regardless of the numbers in the bank accounts.
When's the last time you got a job from the broke-ass bum "class" you seem to idolize?
And if a rising tide doesn't lift all boats, where the hell'd you get that 3-gigaflop computer?
I agree that the brute force approach will get A grades in o-chem, but don't you think that maybe our doctors ought to learn how to think like scientists?
Hell, yes! They should think like scientists, but they don't. A majority of physicians in the US approve of teaching Intelligent Design alongside evolutionary theory, after all.
Further educational devolution (no pun intended) for doctors will not serve any good purpose. Ever looked at the prescribing information for a drug? How in the world is a doctor supposed to understand all that without a background that includes organic chemistry?
I don't know. If I had a choice between hiring somebody who got a 4-year BSEE the usual way, versus somebody who couldn't afford school but who instead downloaded all the lectures and book.PDFs and absorbed equivalent knowledge from those, I'd take the autodidact any day of the week. That's how you hire the next Wozniak.
It does get complicated. Sometimes the band retains the rights to release individual tracks on their own, or reuse them in other projects. Sometimes a label joins the RIAA consortium, or leaves it. Finally, RIAA members sometimes go to a lot of trouble to obscure their membership status.
RIAA Radar should be seen as a helpful guide and not as a religious document, because it's not guaranteed to be 100% accurate.
Slashdot Mirror
I wouldn't argue in favor of serving static, linearly dependent and linearly organized web pages with a UDP-based custom layer. I wouldn't argue against it, but it's not one of those cases where you just slap your forehead at the stuff TCP does to get in your way.
There's a good chance that if you set out to design a transport layer for HTTP you'd end up reimplementing TCP badly, as others in the thread have pointed out. But it's not 1988 anymore, and modern networked applications do a lot of stuff that they didn't do back then. There's no need to reinvent the wheel, but there's also no need to put wheels on a jetpack.
All running applications using the link together know better than TCP/IP how to allocate bandwidth but no one application does.
Gee. <scratches head> If only there were a way for multiple clients to optimize their traffic patterns on a cooperative basis. Something like, perhaps, a network of some sort.
It could be made out of, like, tubes.
I'm obviously not going to write a treatise on RFC 2581 in a Slashdot comment. :-P Suffice it to say that the decision to employ most of those methods -- slow start, congestion avoidance, fast retransmit and fast recovery -- belongs in the app because it simply cannot be optimally implemented by a content-agnostic layer. (If your network depends on well-defined slow start behavior... well, that's a shame, isn't it. Light up some more fiber and send Cisco some more bucks.)
And yes, most of that RFC discusses how, when, and how often to naively retransmit data that the application probably doesn't care about anymore.
So, phrasing this as something other than a flame, you would suggest that HTTP, FTP, etc. use UDP and then let each client application implement its own rate limiting
Because my application is better at knowing how to adapt to a given amount of bandwidth than TCP/IP is.
retransmission policy
Because non-trivial applications often have better ways to deal with packet loss than TCP's acknowledgement/retransmission mechanism. ...and packet reordering?
It often makes more sense to let your application deal with this. BitTorrent's a great example, as are most action games that do client-side prediction. Likewise, the way streaming applications have to work in the presence of TCP's reordering is by grinding to a halt waiting for the retransmission. Gee, thanks, TCP. Instead of a 16-millisecond glitch, I get a 2000-millisecond dropout.
Or did you have some other exotic transport layer in mind to replace TCP
No; I believe that dictating those details to application developers is counterproductive. As long as the implementation is competent, datagram transport with application-specific reliable delivery is almost always a win for everybody concerned -- clients, servers, routers. And if the implementation is incompetent, TCP can only do so much to help.
What? Why? Why is a connection based application "bad design"?
Because connection maintenance is a very high-level task that should not be delegated to the very bowels of the application, much less to someone else's opaque API.
A "session" is just a hack to give you connection-like qualities over a connectionless protocol, such as HTTP. If you want connection-like behavior, and you're free to design your own protocol, why not just use connections? Why make your life more complicated?
The complication comes in when you have to code "connected versus unconnected" states into your app all the way up to the user-interface level, and you realize that you're doing a lot of redundant work to try to keep track of what TCP/IP thinks the connection state is. Admittedly, it's hard to explain unless you've been there.
There are plenty of advantages to connection-based applications; no need to re-authenticate on every message, for one.
TCP is doing this behind your back. Maybe you can do it better.
Clear indications of when a client is done talking to you for another (the connection is dropped, whereas in a connectionless world you never really know if the client has died or is just about to send you another message).
That's one of the worst examples you could pick. When, and whether, TCP will drop a connection is up to factors far beyond the application's control. You have to handle the "TCP dropped connection" state anyway; it's not really any easier to do that at the lowest networking level than it is to do it at a higher level.
Most applications I've worked on have ended up with their own keepalive-based housekeeping mechanisms anyway; the obvious tactic is to use those mechanisms for connection maintenance instead of a low-level TCP connection whose persistence your own code has to mirror.
Even most "connectionless" applications, such as your web browser, try to cheat and get connection based behavior.
Again you're making my point for me. Because TCP-level connections are not appropriate for Web browsing, various Procrustean hacks get brought into play.
Renegotiating an SSL connection for every request would be remarkably expensive for the server, so the server and browser try to reuse the same connection as much as possible. Fire up wireshark, and you'll see your browser will only open two connections when you read Slashdot, and all the images and text all come across those two connections.
Sure, but that has nothing to do with any inherent advantages of TCP-level connections, except insofar as SSL is tied to them. If you were driven by absolute maximization of server performance and weren't saddled with the baggage of existing protocols like HTTP, you certainly wouldn't use TCP; it'd be like doing unnecessary work in the inner loop of a graphics routine. That's the decision the BitTorrent people are faced with.
TCP does work well, and certainly the axiom about those who don't understand TCP being condemned to reimplement it badly is a valid one. However, it is the wrong tool for many modern applications. If you're writing networking code and you find yourself fighting TCP's behavior, it's not necessarily your fault.
The developers of BitTorrent have long since passed the level of play where they're better off using TCP/IP because "well, because that's what you're supposed to use." My objection was to the naive canonization of TCP as an all-purpose protocol that's somehow magically superior to any protocol that the application can implement on its own behalf.
"Because it's the most router-friendly protocol" is also no excuse. Applications that aren't a good fit for TCP often abuse the protocol, e.g. by disabling Nagling and tinkering with other parameters in an effort to make it work more like, well, UDP.
Those developers didn't reimplement TCP badly, but they might as well have, as their efforts are likely to have the same effect.
To say that TCP is optimal at dealing with congestion is to say that individual packets are always a good representation of the data blocks being sent and received at the application level, and that best thing that any application can do when expected data doesn't arrive is to wait on it to be retransmitted, with the network layer queuing up all subsequent intact packets.
Once again, this behavior is guaranteed to be completely wrong for anything but toy command-line applications that fit on a single page in the back of a musty-smelling manual.
There's no way that BT's protocol could be as sophisticated as TCP, given its 30+ years of development. Most people don't appreciate how amazingly well TCP's flow control works in terms of maximizing link utilization in a way that is fair to all network users. We really don't need is an arms race of new, greedier protocols.
TCP gets a lot of credit it doesn't deserve. It enforces bad design -- most client/server applications should be either stateless or session-based, rather than connection-oriented. Anything that even vaguely resembles a streaming application shouldn't even consider TCP. Finally, TCP's connection model is almost guaranteed to be suboptimal for any application that does require one.
What are the odds that HTTP, FTP, SMTP, and BitTorrent will all work optimally over TCP? Slim to none, and none is still waiting for Nagle.
So how's life at Shure these days? You guys hiring?
Don't know what John Cleese has to say about it, but Jorge of Burgos sure seems pissed.
PDF was never made for 'importing' and editing.
True, but the reality is, users want to be able to do just that.
Developers who satisfy that wish are going to do very well for themselves over the next few years.
I'm not an libertarian in any way, but prohibiting things just because they are dangerous to the same person that chooses to do those things is stupid and harmful to everybody.
That's more or less the definition of "libertarian." If someone isn't either harming you directly or about to do so, let them be.
I didn't exactly respect him more after reading Travels. It was hard to escape the conclusion that Crichton was a guy who would believe literally anything anyone told him. That's one reason I was somewhat surprised to see him arguing in favor of more objective thinking in the global-warming debate. His bio made him sound like a real New Age woo-woo type.
Make no mistake, the american dream is the ultimate enslavement tool. Because the truth of course is that NOT everyone can make it, no matter how they try because a capatalist system needs its homeless to allow for the superrich. The american dream at best is an lottery, but one where the winners can buy the winning tickets.
Funny, though, everybody who tries to improve on this system ends up killing tens of millions of their own people. Any suggestions for avoiding that?
The cryptographic term of the day is "One-Time Pad," boys and girls. Can you say "One-Time Pad?"
Translation: "I have a Level 74 Hermaphroditic Fighter/Mage/Sous Chef character in World of Elfcrap, why would I ever go outside?"
<shrug> I'm not a political guy, really. Just wanted to point out to the original poster that the time to get all panicky about "creeping socialism" was 70 years ago. It is way too late now, for better or for worse. These days, the debate on socialistic government can only address "how much," not "if" or "when."
Congratulations in reading a value judgment into my post that wasn't there. Now go call your doctor -- private, socialized, or otherwise -- about that twitching knee.
Lets start at the beginning. Grandparents on mother side were both communists. Mom was at least an anti-american, probably socialist leaning if not a fellow traveller. Dad? Communist. What else do you call a man who gets tossed out of an overtly Socialist government for being to much of a redistributionist? Growing up Grampa figures Young Obama needs a proper father figure who can teach him how to be black enough to properly hate on whitey so he fixes him up with his old CPUSA buddy Frank Marshal Davis. Poor kid never really stood a chance, but then EVERYBODY at some point has to decide whether to keep the morals they grew up with or not.
What will really bake your noodle is if, like me, you ever run across an old socialist propaganda tract from the 1930s era in a used-book store. You will feel like you took the red pill with an LSD chaser; you will see that the agitators among Obama's ancestors won out in the end. They have gotten 98% of everything they ever asked for, from Social Security, to expansion of Federal oversight over the banking and financial sectors, to legal recognition for organized labor, to racial and sexual parity.
It's stunning when you realize that we are not only staring down a socialist path as a nation, but we've been on one for the last 80 years. There truly is very little new under the Sun. Obama likely represents nothing more than the status quo... which ironically makes him the real Conservative.
The Republicans running these days are a bunch of nation-building Jesus freaks who have long abandoned any claim to conservative ideology. (I mean, really. How else can a Republican define "epic fail" than to be kicked out of government in favor of an attractive and successful African American named Hussein?!)
I say let's elect someone from the right side of the bell curve for a change, just to see what happens.
I could make about $120,000 per hour as an Armored Car Guard, but the retirement plan sure sucks.
I can never understand the cadre of poor and middle class who feel the need to defend the rich. Do you think defending the rich will make you rich? Do you think the rich will accept you as anything but a quaint and loyal pet? Or maybe you feel the tax burden, and are just pissed off about taxes? If that's the case, don't you realize that you, the poor or middle class, are paying an unfair burden at the expense of the wealthy?
Nor can I ever understand people who divide other people into Manichaean classes like you're doing. Economic policies are either sound or unsound, regardless of the numbers in the bank accounts.
When's the last time you got a job from the broke-ass bum "class" you seem to idolize?
And if a rising tide doesn't lift all boats, where the hell'd you get that 3-gigaflop computer?
Yeah, whatever. Either way, less scientific education is clearly not what's called for.
I agree that the brute force approach will get A grades in o-chem, but don't you think that maybe our doctors ought to learn how to think like scientists?
Hell, yes! They should think like scientists, but they don't. A majority of physicians in the US approve of teaching Intelligent Design alongside evolutionary theory, after all.
Further educational devolution (no pun intended) for doctors will not serve any good purpose. Ever looked at the prescribing information for a drug? How in the world is a doctor supposed to understand all that without a background that includes organic chemistry?
I don't know. If I had a choice between hiring somebody who got a 4-year BSEE the usual way, versus somebody who couldn't afford school but who instead downloaded all the lectures and book .PDFs and absorbed equivalent knowledge from those, I'd take the autodidact any day of the week. That's how you hire the next Wozniak.
It does get complicated. Sometimes the band retains the rights to release individual tracks on their own, or reuse them in other projects. Sometimes a label joins the RIAA consortium, or leaves it. Finally, RIAA members sometimes go to a lot of trouble to obscure their membership status.
RIAA Radar should be seen as a helpful guide and not as a religious document, because it's not guaranteed to be 100% accurate.