Slashdot Mirror
Taking Google's QUIC For a Test Drive
agizis writes "Google presented their new QUIC (Quick UDP Internet Connections) protocol to the IETF yesterday as a future replacement for TCP. It was discussed here when it was originally announced, but now there's real working code. How fast is it really? We wanted to know, so we dug in and benchmarked QUIC at different bandwidths, latencies and reliability levels (test code included, of course), and ran our results by the QUIC team."
Javascript required to view *static* content?
No.
Learn how to write a webpage properly.
...but Google said something.
Let's fix it!
Right now, seems that QUIC is not that quic.
I understand the limitations of TCP, and although QUIC may look good on paper, the benchmarks provided in the link provided show that in every test QUIC failed miserably and was far worse than TCP. So the real-world benefits of QUIC would be what then? Once Google has a protocol that actually out-performs the tried and true on every front then bring it to the party, otherwise just stahp already.
Paragraph 1 of RFC:
User SHALL register for Google Plus
Wow, those slides are HIDEOUS. I'm guessing that's one of the default themes for the Google Apps slideshow thing... it's gotta be one of the worst, though.
The current problem with UDP is that many border routers do not check whether outgoing udp packages are from within their network. This is the base for DNS based ddos attacks. They are very difficult to mitigate on server level without creating openings for Joe job attacks instead... Standardizing on udp for other protocols will emphasize this problem
If an experiment works, something has gone wrong.
The Internet has needed a standardized reliable UDP protocol for many years. There have been many attempts, but hopefully this one will stick.
UDP is for messages (eg. DNS) and real time data. TCP is far superior for bulk data because of the backoff mechanism, something they want to work around with that UDP crap.
QoS works perfectly with TCP because of TCP backoff.
So much wrong with this idea it makes my head hurt. It is OK to run game servers with UDP. It is OK for RT voice or even video to use UDP. It is not OK to abuse the network to run bulk, time insensitive transfers over UDP, competing with RT traffic.
What is the problem? Too many connections for 1 resource each? Too many TCP handshakes? You know, there was this idea about PIPELINING. Perhaps improve that a little to reduce 200 tcp connections to maybe 3 or 5, instead of piping data over UDP. But that would be too easy - why not invent your own network stack instead :S
What are people at Google smoking?
The problem with that UDP proposal is that the IETF may not get it.
As someone working with the project.
The benchmarking here is premature.
The code is not yet implementing the design, it is just barely working at all.
Again, they're not (yet) testing QUIC-- they're testing the very first partial implementation of QUIC!
That being said, it is great to see that others are interested and playing with it.
I don't care about the speed just yet. Tell me how this screws with my privacy first.
Hmmm, several posts, yet no mention of UDT so far. It would be nice if the benchmark included it.
`echo $[0x853204FA81]|tr 0-9 ionbsdeaml`@gmail.com
As a newish developer who knows only the minimum I need to about TCP/IP protocol, I was surprised that this, and a number of common things (apparently games, streaming video) use UDP at all. I thought it was basically just used for ping.
Out of curiosity can anyone point out good books for learning more about how to implement applications that use TCP/IP including udp in ways other than the common ssh/http/ftp connections.
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
QUIC gets all of those page elements at the same time, over a single connection. The problem with TCP and the strength of QUIC is exactly what TFA chose NOT to test. By using a single 10 MB file, their test is the opposite of web browsing and doesn't test the innovations in QUIC.
* browsers can negotiate multiple TCP connections, which is a slow way to retrieve many small files.
Yes, you're absolutely right that this left out stream multiplexing, but it did test QUIC's ability to handle packet loss. Seeing as how QUIC is aiming, at a high level, to fix problems in TCP and make the Internet faster, I think the test is fair, and I'm excited to see how things improve. There are other scenarios in the tests in Github, including some sample webpages with multiple images and such, if anyone is interested.
Thank for that info, and for making your test scripts available on Github.
I'm curious* what were the results of web page tests? Obviously a typical web page with CSS files, Javascript files, images, etc. is much different from a monolithic 10 MB file.
* curious, but not curious enough to run the tests for myself.
Those fuckers at www.connectify.me redirected my connection attempt to
http://www.connectify.me/no-javascript/ so that even after I authorized Javascript for their site I was unable to navigate to my intended destination (whatever shit they pulled did not even leave a history item for the originally requested URL).
This sucks because I middle-click many URLs into tabs I might not visit until ten minutes later. It I had a bunch of these tabs open I wouldn't even have been able to recollect where I had originally been. In this case, I knew to come back here.
Those fuckers at www.connectify.me need to procure themselves an Internet clue stick PDQ.
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
QUIC gets all of those page elements at the same time, over a single connection. The problem with TCP and the strength of QUIC is exactly what TFA chose NOT to test. By using a single 10 MB file, their test is the opposite of web browsing and doesn't test the innovations in QUIC.
* browsers can negotiate multiple TCP connections, which is a slow way to retrieve many small files.
What the hell are you talking about? You're conflating HTTP with TCP. TCP has no such limitation. TCP doesn't deal in files at all.
a handful of selfish greedy people are no match for millions of selfish, greedy people -u4ya
May I be so bold as to suggest graphs citing only x performance improvement for protocol y are insufficient, harmful and useless measures of usable efficiency. We know how to make faster protocols.. the challenge is faster while preserving generally meaningful congestion avoidance. This part is what makes the problem space non-trivial.
Look at TFA and connectify links it is all performance talk with total silence on addressing or simulating congestion characteristics of the protocol.
Having sat in on a few tcpm meetings it is always the same with google... they show data supporting by doing x there will be y improvement but never as much enthusiasm for consideration of secondary repercussions of the proposed increased network aggression.
My personal view RTT reductions can be achieved thru extension mechanisms to existing protocols without wholesale replacement. TCP fast open and TLS extensions enabling 0 RTT requests thru the layered stack...experimental things for which "code" exists today can provide the same round trip benefits as QUIC.
What google is doing here is taking ownership of the network stack and congestion algorithms away from the current chorous of stakeholders and granting themselves the power to do whatever they please. No need to have a difficult technical discussion or get anyones opinions or signoff before droping in a new profit enhancing congestion algorithm which could very well be tuned to prefer google traffic globally at the expense of everyone elses ... they control the clients and the servers...done deal.
There are two fundamental improvements I would like to see regarding TCP.
1. Session establishment in the face of in band adversaries adding noise to the channel. Currently TCP connections can be trivially reset by an in-band attacker. I think resilience to this necessarily binding security to the network channel can be a marginally useful property in some environments yet is mostly worthless in the real world as in-band adversaries have plenty of other tools to make life difficult.
2. Efficient Multi-stream/message passing. Something with the capabilities of ZeroMQ as an IP layer protocol would be incredibly awesome.
Well they should at least wait for a beta rather than alpha.
Any sufficiently simple magic can be passed off as mere advanced technology.
Benchmarking for the sake of bug testing should happen al the time.
Benchmarking for the sake of a story could be useful. That is what Phoronix is all about, for the most part.
But, testing for your own use cases, as these guys say they are testing in TFA, before the developers say it is even implemting the actual design is pointless.
To then publish the results is simply moronic.
> You're conflating HTTP with TCP.
I'm discussing how HTTP over TCP works, in contrast to how it works over QUIC.
TCP provides one stream, which when used with HTTP means one file.
QUIC provides multiple concurrent streams specifically so that http can retrieve multiple concurrent files.
I haven't RTFA and I don't know much about QUIC, but if it's what you suggest...
As I understand it, QUIC is largely about multiplexing - downloading all 35 files needed for a page concurrently. The test was the opposite of what QUIC is designed for
TCP handles one file at at a time* - first download the html, then the logo, then the background, then the first navigation button ....
...then it sounds like a really horrible idea. If I click on a link will I have to wait for 20MB of images to finish downloading before I can read the content of a webpage, only to find out it wasn't what I was looking for anyway?
I'm no networking expert, but does anyone know how this compares to SCTP?
And have they taken various security considerations into account, e.g. SYN floods?
When I look at the Goodput vs. Bandwidth being capped for QIC, and the statement that most folks don't use more than that, my thought was "so why does it exist in TCP?".
Is this something exploited primarily by hackers but not giving any value to normal humans?
The slides refer to a feature called "pacing" where it doesn't send packets as fast as it can, but spaces them out. Can someone explain why this would help? If the sliding window is full, and an acknowledgement for N packets comes in, why would it help to send the next N packets with a delay, rather than send them as fast as possible?
I wonder if this is really "buffer bloat compensation" where some router along the line is accepting packets even though it will never send them. By spacing the packets out, you avoid getting into that router's bloated buffer.
From the linked slides:
Does Packet Pacing really reduce Packet Loss?
* Yes!!! Pacing seems to help a lot
* Experiments show notable loss when rapidly sending (unpaced) packets
* Example: Look at 21st rapidly sent packet
- 8-13% lost when unpaced
- 1% lost with pacing
I think I'll wait for QUICv6, thanks.
Mastering the English language is fucking easy: all you have to do is to put an f* word in every fucking sentence.
I tend to agree. I am glad that someone is trying to create a better TCP. If they fail, we validate that TCP is a good idea. If they succeed, then we can have a better protocol.
If the QUIC exercise is successful, then the IETF should consider extending TCP to support the relevant features. For example, their point about multiple steams is a good one. Perhaps TCP should define an option to open N simultaneous connections with a single 3-way handshake. Existing implementations would ignore the new option bytes in the header so nothing would break.
If adding FEC really helps, then the same logic applies. Add it. If pacing really helps, then when you get an ack of 10 packets, pace the replies (although I doubt that would really help). But let us add these things only once they are proven to work.
You make a good point about poor congestion control causing wide-scale harm, but that problem exists today. Anyone can create any protocol they want, as long as it is built on TCP or UDP. This has always been the case and so far no one has managed to create a protocol that destroys the internet... yet. :-) Although no one as powerful as Google has tried. And certainly some protocols have brought down LANs due to inefficiency (Windows file sharing).
So then don't you just need http to transfer multiple files at the same time over 1 stream?
this is like replacing ethernet with something more reliable because dns queries fail over udp sometimes. You're wrenching on the wrong protocol.
Are you friggin nuts? This seems to imply that any filtering at the kernel level will need to unwrap all the application specific jibber-jabber in this protocol to determine wtf it's supposed to do with it. That would be quite costly in terms of performance. No, I don't trust applications to handle the security of packet data coming in. Especially when some entity wants to bury support for the protocol in their own web browser. This just smells like all kinds of Hell naw .
Join the Slashcott! Feb 10 thru Feb 17!
The benchmarking itself is awesome-- it is good to have people playing with it.
Concluding things right now, when the implementation is working towards correctness instead of optimality, however, is potentially misleading.
If the QUIC exercise is successful, then the IETF should consider extending TCP to support the relevant features. For example, their point about multiple steams is a good one. Perhaps TCP should define an option to open N simultaneous connections with a single 3-way handshake. Existing implementations would ignore the new option bytes in the header so nothing would break.
While TCP is ancient there has been continuous work to improve it over the years. I think most people throwing stones here have not taken the time to look around and understand the current landscape. Indeed many ideas in QUIC are good ones yet not a single one of them are something new or something that had not been implemented or discussed in various WGs.
Regarding multiple streams what effectively is the difference between this and fast open? I send a message the message arrives and is processed immediately without a round trip... with a scenario like that what is even the point of something labeled "multi-stream"?
You make a good point about poor congestion control causing wide-scale harm, but that problem exists today. Anyone can create any protocol they want, as long as it is built on TCP or UDP.
TCP congestion is normally implemented within the operating system where user space does not have access or visibility into scheduling of transmission.
UDP mostly used for low bandwidth exchanges or inherently rate limited realtime communications. Most of these applications sport non-existent or insufficient congestion coverage. For example a bit torrent client without built in queue management or where you have neglected to set rate limits will easily saturate any network link to the point of being unusable. UDP congestion is a problem it just is not that big of a bandwidth consumer to have a large scale impact where it can be a threat to the network.
To be clear I don't think Google is at risk of bring about a congestive apocalypse. The potential risk I see takes the form of excessive aggression for competitive advantage.
The benchmark looked well constructed, and as such is a fair test for what it is testing: unfinished-userspace-QUIC vs kernel-TCP
It will be awesome to follow along with future runs of the benchmark (and further analysis) as the QUIC code improves.
It is awesome to see people playing with it, and working to keep everyone honest!
AC, don't worry.
TCP is simply a reliable, in-order stream transport.
HTTP on TCP is what was described, and, yes, not the best idea in today's web (though keep in mind that most browsers open up 6 connections per hostname), but that is also why HTTP2 is working on being standardized today.
Nah; it is valuable for many people to be doing this benchmarking even with the current state of code. .. it just requires careful explanation of what the benchmark entails!
Concluding that buggy-unfinished-QUIC is slower than TCP is absolutely valid, for instance.
That isn't the same as QUIC being slower than TCP (at least, not yet!)
Actually not quite.
IP provides a bnuch of packets: there is no port number.
TCP and IP provide multiplexing over IP by introducing the port number concept.
SJW n. One who posts facts.
> reliable UDP protocol You want a reliable *unreliable* datagram protocol protocol? Sounds like something guaranteed to fail. Everyone tries to reinvent TCP. Almost always they make something significantly worse. This is no exception.
I once worked at a company that made Parking Meters - and accepted credit cards at them. They sent their data over https, and had random issues with timeouts.
It turns out they would format their data in (very descriptive) XML, and discovered an excessively large file combined with an SSL handshake over crappy 2g connection took too long to transfer the data (it didn't help the programmers 'forgot' they hardcoded a timeout, so if the comms was just slow, it would throw a generic error and they blamed Apache for it).
In any case, the offshore dev team's solution was to create a UDP client/server protocol of their own.
It was working nicely when I left, and was PCI Compliant, but at that point we had no way to reliably monitor communications from the perspective of the meter because we (SysAdmins in charge of the backend systems) would have had to write proprietary code from non-existing documentation just to replicate what used to be a simple HTTP POST.
Some things look great, but aren't thought out all the way ...
"I can't give you a brain, so I'll give you a diploma" - The Great Oz (blatently stolen sig)
One does wonder though how it would compare vs. an extension to keepalive that truly multiplexes OR that allows queueing requests in the event that the web server might be able to do something useful if it knows what it will be sending next.
There may be cases where FEC is useful at the protocol layer, but it seems like the link layer is amore likely to be the right place for it. The link knows what sort of link it is and how likely it might be to lose data. That would also mean that if there are multiple lossy links, the error correction would be refreshed at each hop and so gain effective reliability.
The more interesting idea is using uuids to permit continued communication as a mobile device hops from one network to another. That can theoretically be done at a lower level but given the least common denominator behavior of providers, that won't happen in practice.
I certainly don't mind is someone wants to play around with protocols though.
Sadly ports are dead, and we're watching them get re-invented. We've gone from the web being a service built on the internet, to the internet being a service build on the web (well, on port 80/443). Security idiots who mistook port-based firewalling for something useful have killed the port concept, and now that we're converging on all-443-all-the-time, we have to re-invent several wheels.
Socialism: a lie told by totalitarians and believed by fools.
Yes, you just need to redo http. While you're redoing http, you make several improvements.
As you improve http, you realize the biggest performance issues for http come from the fact that it's limited by the requirement that sends and recieves via an ancient protocol that wasn't designed to carry http. Http doesn't run atop TCP because it's a good fit - on runs atop http because that's what was available.
I think it's more like designing automobiles 3.0, designed to go 300 MPH, and realizing that if you want 300 MPH vehicles, you might need to make some significant changes to highway design.
Yeah, I assume that's obvious. Send the html first, then css, js, and images concurrently.
Also, I hope you don't browse too many pages with 20MB of images. :)
You know, there was this idea about PIPELINING.
Yes, it'd be nice if they'd take that idea a little more seriously. Some years ago I looked into it and it makes a hell of a difference.
The basic problem that pipelining solves is that even with keep-alive connections, you've got a minimum download time equal to your round-trip ping time for each file you download. If you visit a web page, and it has 100 images, and your ping time to the site is 500 ms because it's on the other side of the planet, you're going to wait 50 seconds to download that page over a single HTTP connection regardless of how small the files are. This is because the web browser requests one file, then waits to receive it before requesting the next file. Pipelining solves the problem by allowing the web server to say "hey, go ahead and send all the requests at once if you want to." It's also trivial to implement, in most cases requiring no changes whatsoever, as the web server has no interest in reading from the socket until it's responded to the previous request anyway. The idea that you wouldn't be able to just send all of the requests immediately is actually kind of silly.
At the time I tested it, I found it to be far faster than my web browser making multiple simultaneous connections to a test web site I set up, although my test was a bit incomplete: I was comparing the time the web browser took to load the page to the time it took me to receive all of the data after sending all of the requests over a single HTTP connection, as I couldn't find a web browser at the time that supported pipelining and so I just piped all the request headers through the 'hose' command. (The problem with that test is that it ignores time the web browser spends doing other things, like decompressing JPEG images, which is far from trivial.) Never the less, I was comparing a web page that took 30 seconds to load over a single connection to receiving all of the data almost instantly, so I'm pretty sure most of the improvement I saw was from utilizing pipelining.
I support your views that the selfish speed measurements in TFA fail to capture the design constraints of a TCP replacement, but I don't think your characterization of Google's involvement in the tcpm list is a fair one. They are not at all ignorant of the design constraints, as you can see from their paper. The tcpm threads I've read look more like heckling bike-shedders who, at best, may have done a little research a decade ago, and are now directing their intellectual efforts toward being obstructionist to the only people left doing real research. What TCP research is there outside Google, which is practical, and useful to the web? The other problem is the baseline of strangling conservatism founded from a self-aggrandizing position of "responsibility," which is not consistent with the track record of this area of research: overconservatism probably played a part in the congestion-collapse of the Internet in the 80's which you arrogant eggheads then scrambled to fix. It was not the case that the default resistance to any change in TCP saved the internet, nor is it remotely true that we're unaware of risky corner-cases existing in TCP now that most proposals aim primarily to smooth out, nor is it even true we're not beginning to see the effects of these corner-cases to an extent that could predict another collapse-like event. How about, this time, we respond to the very real problems we see on the web: video doesn't work, bandwidth isn't fairly divided in the specific low-hanging-fruit case of multiple streams unordered with respect to one another, cellular networks discard half their packets under congestion, TCP adds too much jitter over the network when new applications (even web browsing) benefit from much less jitter than we're seeing. I understand and agree that deconstructionism has some value, but Google isn't the only one risking being "bad for the Internet." You, sir, also risk that, and honestly at this point I trust Google not to break the Internet more than I trust the get-off-my-lawn retirees on tcpm that claim to exist only for said purpose.
You make a good point about poor congestion control causing wide-scale harm, but that problem exists today. Anyone can create any protocol they want, as long as
I disagree with the parent, but not on this basis. Technically-sophisticated hobbyists being able to use any UDP protocol they want is a completely different level of risk to the Internet than the hypotheticals of pushing a new protocol to Chrome stable channel, or to TCP in a Mac OS update, or to Youtube/Netflix/Amazon servers.
Doesn't IPSEC protect against this?
Isn't that what SCTP is for?
Yup; True