Slashdot Mirror
Taking Google's QUIC For a Test Drive
agizis writes "Google presented their new QUIC (Quick UDP Internet Connections) protocol to the IETF yesterday as a future replacement for TCP. It was discussed here when it was originally announced, but now there's real working code. How fast is it really? We wanted to know, so we dug in and benchmarked QUIC at different bandwidths, latencies and reliability levels (test code included, of course), and ran our results by the QUIC team."
Javascript required to view *static* content?
No.
Learn how to write a webpage properly.
Yeah, with the current state of the code, and for the scenarios we tested, that's about right. Google's big focus is on doing a better job of multiplexing streams and reducing the amount of round-trips required to establish the connection and stuff. So our test scenario, of pulling down a 10MB psuedorandom file, is a scenario that's near and dear to our hearts, but isn't at the top of Google's TODO. I suspect that flipping on forward error correction is a simple thing, and changing the maximum congestion window size to better overcome the Long Fat Network problem shouldn't be too bad, either.
It is broken. Google just made a bad solution. Doesnt mean the problem doesnt exist.
Paragraph 1 of RFC:
User SHALL register for Google Plus
The current problem with UDP is that many border routers do not check whether outgoing udp packages are from within their network. This is the base for DNS based ddos attacks. They are very difficult to mitigate on server level without creating openings for Joe job attacks instead... Standardizing on udp for other protocols will emphasize this problem
If an experiment works, something has gone wrong.
The Internet has needed a standardized reliable UDP protocol for many years. There have been many attempts, but hopefully this one will stick.
UDP is for messages (eg. DNS) and real time data. TCP is far superior for bulk data because of the backoff mechanism, something they want to work around with that UDP crap.
QoS works perfectly with TCP because of TCP backoff.
So much wrong with this idea it makes my head hurt. It is OK to run game servers with UDP. It is OK for RT voice or even video to use UDP. It is not OK to abuse the network to run bulk, time insensitive transfers over UDP, competing with RT traffic.
What is the problem? Too many connections for 1 resource each? Too many TCP handshakes? You know, there was this idea about PIPELINING. Perhaps improve that a little to reduce 200 tcp connections to maybe 3 or 5, instead of piping data over UDP. But that would be too easy - why not invent your own network stack instead :S
What are people at Google smoking?
The problem with that UDP proposal is that the IETF may not get it.
As someone working with the project.
The benchmarking here is premature.
The code is not yet implementing the design, it is just barely working at all.
Again, they're not (yet) testing QUIC-- they're testing the very first partial implementation of QUIC!
That being said, it is great to see that others are interested and playing with it.
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
QUIC gets all of those page elements at the same time, over a single connection. The problem with TCP and the strength of QUIC is exactly what TFA chose NOT to test. By using a single 10 MB file, their test is the opposite of web browsing and doesn't test the innovations in QUIC.
* browsers can negotiate multiple TCP connections, which is a slow way to retrieve many small files.
I think most of the people screaming "TCP is broken!" are those with lots of bandwidth who have very specific uses. TCP seems to work quite good for almost everything I have thrown at it. I have a low latency 500kbps down / 64kbps Internet connection and do mostly SSH and HTTP. I am able to saturate my link quite well. I don't know if the QUIC guys are thinking about a significant portion of the population who simply can't get >1.5Mbps connectivity to their homes. In fact their slides seem to indicate they are more wasteful of bandwidth - a risky tradeoff IMHO.
Yes, you're absolutely right that this left out stream multiplexing, but it did test QUIC's ability to handle packet loss. Seeing as how QUIC is aiming, at a high level, to fix problems in TCP and make the Internet faster, I think the test is fair, and I'm excited to see how things improve. There are other scenarios in the tests in Github, including some sample webpages with multiple images and such, if anyone is interested.
Those fuckers at www.connectify.me redirected my connection attempt to
http://www.connectify.me/no-javascript/ so that even after I authorized Javascript for their site I was unable to navigate to my intended destination (whatever shit they pulled did not even leave a history item for the originally requested URL).
This sucks because I middle-click many URLs into tabs I might not visit until ten minutes later. It I had a bunch of these tabs open I wouldn't even have been able to recollect where I had originally been. In this case, I knew to come back here.
Those fuckers at www.connectify.me need to procure themselves an Internet clue stick PDQ.
As a newish developer who knows only the minimum I need to about TCP/IP protocol, I was surprised that this, and a number of common things (apparently games, streaming video) use UDP at all. I thought it was basically just used for ping.
Out of curiosity can anyone point out good books for learning more about how to implement applications that use TCP/IP including udp in ways other than the common ssh/http/ftp connections.
ICMP is used for ping, friend. I recommend the Comer books. Also, I'd also recommend that you read the IP, UDP and TCP specs.
No, no, you're not thinking; you're just being logical. --Niels Bohr
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
QUIC gets all of those page elements at the same time, over a single connection. The problem with TCP and the strength of QUIC is exactly what TFA chose NOT to test. By using a single 10 MB file, their test is the opposite of web browsing and doesn't test the innovations in QUIC.
* browsers can negotiate multiple TCP connections, which is a slow way to retrieve many small files.
What the hell are you talking about? You're conflating HTTP with TCP. TCP has no such limitation. TCP doesn't deal in files at all.
a handful of selfish greedy people are no match for millions of selfish, greedy people -u4ya
May I be so bold as to suggest graphs citing only x performance improvement for protocol y are insufficient, harmful and useless measures of usable efficiency. We know how to make faster protocols.. the challenge is faster while preserving generally meaningful congestion avoidance. This part is what makes the problem space non-trivial.
Look at TFA and connectify links it is all performance talk with total silence on addressing or simulating congestion characteristics of the protocol.
Having sat in on a few tcpm meetings it is always the same with google... they show data supporting by doing x there will be y improvement but never as much enthusiasm for consideration of secondary repercussions of the proposed increased network aggression.
My personal view RTT reductions can be achieved thru extension mechanisms to existing protocols without wholesale replacement. TCP fast open and TLS extensions enabling 0 RTT requests thru the layered stack...experimental things for which "code" exists today can provide the same round trip benefits as QUIC.
What google is doing here is taking ownership of the network stack and congestion algorithms away from the current chorous of stakeholders and granting themselves the power to do whatever they please. No need to have a difficult technical discussion or get anyones opinions or signoff before droping in a new profit enhancing congestion algorithm which could very well be tuned to prefer google traffic globally at the expense of everyone elses ... they control the clients and the servers...done deal.
There are two fundamental improvements I would like to see regarding TCP.
1. Session establishment in the face of in band adversaries adding noise to the channel. Currently TCP connections can be trivially reset by an in-band attacker. I think resilience to this necessarily binding security to the network channel can be a marginally useful property in some environments yet is mostly worthless in the real world as in-band adversaries have plenty of other tools to make life difficult.
2. Efficient Multi-stream/message passing. Something with the capabilities of ZeroMQ as an IP layer protocol would be incredibly awesome.
The general gist is that UDP and TCP both have kind of an ideal mileu. UDP is great for small packets that you want delivered with a minimum of overhead, and if the packet is late, lost, or out of order, it won't kill anything.
TCP is great if you are sending large amounts of data at once, between a pair of systems, and in situations where its important not to lose packets or get them out of order, and you don't care that much if this takes a little extra time (occasionally perhaps a lot of extra time) to accomplish. Also good in situations where you'd like to know when your partner on the other side goes away for some reason.
Most applications are going to be in-between somewhere, so you have to make a decision. For example, if your packets are small and need to be delivered quickly, but you also need reliability, you might go with TCP just to get that reliability. Alternatively, if you can get away with it, you might instead go with UDP, but use dedicated links between the systems and a handshaking protocol at your application layer to prevent collisions.
Alternatively, you might do what Google is doing, and try to reimplement TCP's reliability in your application layer on top of UDP. The thing about UDP is that you can always reimplement any parts of TCP you need on top of it.
In some of the web page scenario tests, HTTP over QUIC was about 4x faster than HTTP over TCP, and in others, QUIC was about 3x worse. I'll probably look into that next. The QUIC demo client appears to take about 200ms to get warmed up for a transfer, so testing with small files over fast connections isn't fair. After that 200ms, it seemed to perform as one would expect, so tests that take longer than a couple seconds are a pretty fair judge of current performance.
I haven't RTFA and I don't know much about QUIC, but if it's what you suggest...
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
...then it sounds like a really horrible idea. If I click on a link will I have to wait for 20MB of images to finish downloading before I can read the content of a webpage, only to find out it wasn't what I was looking for anyway?
The slides refer to a feature called "pacing" where it doesn't send packets as fast as it can, but spaces them out. Can someone explain why this would help? If the sliding window is full, and an acknowledgement for N packets comes in, why would it help to send the next N packets with a delay, rather than send them as fast as possible?
I wonder if this is really "buffer bloat compensation" where some router along the line is accepting packets even though it will never send them. By spacing the packets out, you avoid getting into that router's bloated buffer.
From the linked slides:
Does Packet Pacing really reduce Packet Loss?
* Yes!!! Pacing seems to help a lot
* Experiments show notable loss when rapidly sending (unpaced) packets
* Example: Look at 21st rapidly sent packet
- 8-13% lost when unpaced
- 1% lost with pacing
UDP and TCP have different uses; one isn't better than the other, they just do different things.
UDP is message-based. When I send a UDP message the remote end either gets the whole message or none of it. This can make parsing in applications a lot easier; rather than putting delimiters into a stream and trying to pick apart the data as it comes in in chunks I can be sure that I'm always working with a complete message, and the messages can by of different types/sizes/etc. as dictated by my application-layer needs. But there's a maximum size for messages, and if I need to send more data than fits in a single message it's up to my application to ensure they get put into the right order when they are received.
UDP is unreliable, in that if a UDP packet gets drop the message is lost and no notification is made to the sender. Often this is bad, but in certain instances it is valuable. One such instance is data with a short lifetime, such as games or streaming media. If I'm in the middle of a game and a packet gets dropped it doesn't do me any good to get that packet 2 seconds later -- the game has moved on. This unreliable nature also makes UDP simpler; there's no need to setup a "connection" to send UDP data -- you just slap an IP address on the packet and send it along, and the other end will get it or not and use it or not and you don't have to care. So if you're writing a server that will handle billions of clients UDP has a lot less overhead as it doesn't have to keep track of billions of "connections" (or have billions ports available).
TCP is a streaming protocol. You put data in on one end and it pops out in the same order on the other. This is great if you're sending a single file -- you can be sure the other end will get all the bits in the right order. But it also means if you have something important to say you have to wait in line until all of the preceding data has been transmitted, possibly including things that will be expired by the time they are received. It also means your application-layer protocol has to have some method to separate messages if you send more than one thing over a single connection.
TCP has reliable delivery. Often this is a good thing, as the sender can be sure the receiver got all the data (and got it in the right order). But in order to make the protocol reliable the receiver must acknowledge each and every packet from the sender, and the sender and receiver must store information about each other so they can keep track of this ongoing bi-directional connection. So there's at least a couple of round-trip exchanges necessary to setup a TCP connection, and when you connect to a server it must have a free TCP port number for each and every client it's connected to, and enough memory to keep track of all of the connections.
QUIC (and several other new-ish protocols) are proposing a sort of compromise protocol -- a protocol that's both message-based and reliable, and frequently that allows messages of any size. Such protocols provide the delivery assurances of TCP without the waiting-in-line issues the streaming model can produce, and they reduce the amount of setup overhead by allowing clients to open a single connection to the server and fetch many different things.
There are definitely people and opinions on both sides of the fence on this.
Unfortunately, though performance might improve with access to the hardware, wide and consistent deployment of anything in the kernel/OS (how many WinXP boxes are there still??) takes orders of magnitude more time than putting something into the application.
So.. we have a problem: We want to try out a new protocol and learn and iterate (because, trust me, it isn't right the first time out!), however can't afford to wait long periods of time between iterations/availability.
Hopefully the project will drive us all towards solutions to these problems that are generic, usable for any new protocol, and which actually work!
The benchmark looked well constructed, and as such is a fair test for what it is testing: unfinished-userspace-QUIC vs kernel-TCP
It will be awesome to follow along with future runs of the benchmark (and further analysis) as the QUIC code improves.
It is awesome to see people playing with it, and working to keep everyone honest!
AC, don't worry.
TCP is simply a reliable, in-order stream transport.
HTTP on TCP is what was described, and, yes, not the best idea in today's web (though keep in mind that most browsers open up 6 connections per hostname), but that is also why HTTP2 is working on being standardized today.
> reliable UDP protocol You want a reliable *unreliable* datagram protocol protocol? Sounds like something guaranteed to fail. Everyone tries to reinvent TCP. Almost always they make something significantly worse. This is no exception.
I once worked at a company that made Parking Meters - and accepted credit cards at them. They sent their data over https, and had random issues with timeouts.
It turns out they would format their data in (very descriptive) XML, and discovered an excessively large file combined with an SSL handshake over crappy 2g connection took too long to transfer the data (it didn't help the programmers 'forgot' they hardcoded a timeout, so if the comms was just slow, it would throw a generic error and they blamed Apache for it).
In any case, the offshore dev team's solution was to create a UDP client/server protocol of their own.
It was working nicely when I left, and was PCI Compliant, but at that point we had no way to reliably monitor communications from the perspective of the meter because we (SysAdmins in charge of the backend systems) would have had to write proprietary code from non-existing documentation just to replicate what used to be a simple HTTP POST.
Some things look great, but aren't thought out all the way ...
"I can't give you a brain, so I'll give you a diploma" - The Great Oz (blatently stolen sig)
I know a shipping solution, commonly deployed, that already meets the goal of not being worse than TCP in any way. You can probably guess what it is.
Socialism: a lie told by totalitarians and believed by fools.
Sadly ports are dead, and we're watching them get re-invented. We've gone from the web being a service built on the internet, to the internet being a service build on the web (well, on port 80/443). Security idiots who mistook port-based firewalling for something useful have killed the port concept, and now that we're converging on all-443-all-the-time, we have to re-invent several wheels.
Socialism: a lie told by totalitarians and believed by fools.