In some of the web page scenario tests, HTTP over QUIC was about 4x faster than HTTP over TCP, and in others, QUIC was about 3x worse. I'll probably look into that next. The QUIC demo client appears to take about 200ms to get warmed up for a transfer, so testing with small files over fast connections isn't fair. After that 200ms, it seemed to perform as one would expect, so tests that take longer than a couple seconds are a pretty fair judge of current performance.
Maybe, but this still looks really promising. They've made a few really smart decisions, in my opinion.
1) Avoid the usually-doomed-from-the-start approach of starting at the standards committee level. Frame up the project, and let some really smart engineers go nuts. Take what works to a standards committee, and save the time that you would've spent arguing over stuff that might or might not have worked.
2) Make it work with the existing Internet infrastructure. Making everyone adopt a new TCP stack is probably not going to happen. I'm looking at you, Multipath TCP.
3) Do it in the open. Let geeks like me poke around with it so I can complain that it doesn't speed up my file transfers over crappy Internet connections yet.
Yes, you're absolutely right that this left out stream multiplexing, but it did test QUIC's ability to handle packet loss. Seeing as how QUIC is aiming, at a high level, to fix problems in TCP and make the Internet faster, I think the test is fair, and I'm excited to see how things improve. There are other scenarios in the tests in Github, including some sample webpages with multiple images and such, if anyone is interested.
I can take a look at incorporating UDT to the benchmarks going forward. The test scripts are all in github, but I'm not all that familiar with UDT. It looks like UDT4's sendfile/recvfile examples would drop in pretty nicely.
Definitely a work in progress, and the scenario tested is outside of Google's main focus. I suspect that things will get a lot better in the weeks and months to come.
Oh, you should definitely read up on the docs. Yes, they care about backoff mechanisms and generally not breaking the Internet, a LOT. By moving to UDP, they can work on schemes for packet pacing and backoff that do a better job at not overwhelming routers. And they can get away from all the round-trips you need to set up a SSL session over TCP, without sacrificing security.
Its pretty nice that the code is actually in a state that anyone can download, build, and benchmark things they care about, and the stuff presented in the IETF slides (in TFA) is really interesting about how they can use Chrome to A/B test parameters in the protocol, to see which actually work out. Presumably that's just for folks that hop in to chrome internals and enable QUIC, but who hasn't done that already?;)
Yeah, with the current state of the code, and for the scenarios we tested, that's about right. Google's big focus is on doing a better job of multiplexing streams and reducing the amount of round-trips required to establish the connection and stuff. So our test scenario, of pulling down a 10MB psuedorandom file, is a scenario that's near and dear to our hearts, but isn't at the top of Google's TODO. I suspect that flipping on forward error correction is a simple thing, and changing the maximum congestion window size to better overcome the Long Fat Network problem shouldn't be too bad, either.
Yes, +1 to Proxmox. Runs on commodity hardware, performance is good, cluster and backups haven't given me a headache yet. I'm running 100+ VMs across 5 machines, with about a dozen users, and it feels nowhere near its limit.
Slashdot Mirror
In some of the web page scenario tests, HTTP over QUIC was about 4x faster than HTTP over TCP, and in others, QUIC was about 3x worse. I'll probably look into that next. The QUIC demo client appears to take about 200ms to get warmed up for a transfer, so testing with small files over fast connections isn't fair. After that 200ms, it seemed to perform as one would expect, so tests that take longer than a couple seconds are a pretty fair judge of current performance.
Maybe, but this still looks really promising. They've made a few really smart decisions, in my opinion.
1) Avoid the usually-doomed-from-the-start approach of starting at the standards committee level. Frame up the project, and let some really smart engineers go nuts. Take what works to a standards committee, and save the time that you would've spent arguing over stuff that might or might not have worked.
2) Make it work with the existing Internet infrastructure. Making everyone adopt a new TCP stack is probably not going to happen. I'm looking at you, Multipath TCP.
3) Do it in the open. Let geeks like me poke around with it so I can complain that it doesn't speed up my file transfers over crappy Internet connections yet.
Yes, you're absolutely right that this left out stream multiplexing, but it did test QUIC's ability to handle packet loss. Seeing as how QUIC is aiming, at a high level, to fix problems in TCP and make the Internet faster, I think the test is fair, and I'm excited to see how things improve. There are other scenarios in the tests in Github, including some sample webpages with multiple images and such, if anyone is interested.
I can take a look at incorporating UDT to the benchmarks going forward. The test scripts are all in github, but I'm not all that familiar with UDT. It looks like UDT4's sendfile/recvfile examples would drop in pretty nicely.
Definitely a work in progress, and the scenario tested is outside of Google's main focus. I suspect that things will get a lot better in the weeks and months to come.
Oh, you should definitely read up on the docs. Yes, they care about backoff mechanisms and generally not breaking the Internet, a LOT. By moving to UDP, they can work on schemes for packet pacing and backoff that do a better job at not overwhelming routers. And they can get away from all the round-trips you need to set up a SSL session over TCP, without sacrificing security.
Its pretty nice that the code is actually in a state that anyone can download, build, and benchmark things they care about, and the stuff presented in the IETF slides (in TFA) is really interesting about how they can use Chrome to A/B test parameters in the protocol, to see which actually work out. Presumably that's just for folks that hop in to chrome internals and enable QUIC, but who hasn't done that already? ;)
Yeah, with the current state of the code, and for the scenarios we tested, that's about right. Google's big focus is on doing a better job of multiplexing streams and reducing the amount of round-trips required to establish the connection and stuff. So our test scenario, of pulling down a 10MB psuedorandom file, is a scenario that's near and dear to our hearts, but isn't at the top of Google's TODO. I suspect that flipping on forward error correction is a simple thing, and changing the maximum congestion window size to better overcome the Long Fat Network problem shouldn't be too bad, either.
Yes, +1 to Proxmox. Runs on commodity hardware, performance is good, cluster and backups haven't given me a headache yet. I'm running 100+ VMs across 5 machines, with about a dozen users, and it feels nowhere near its limit.