Slashdot Mirror
SPDY Not As Speedy As Hyped?
Freshly Exhumed writes "Akamai's Guy Podjarny reveals after testing: SPDY is different than HTTP in many ways, but its primary value comes from being able to multiplex many requests/responses from client to server over a single (or few) TCP connections. Previous benchmarks tout great benefits, ranging from making pages load 2x faster to making mobile sites 23% faster using SPDY and HTTPS than over clear HTTP. However, when testing real world sites I did not see any such gains. In fact, my tests showed SPDY is only marginally faster than HTTPS and is slower than HTTP."
I use Mycleanpc.bulshit for all my benchmarking needs!
There Can Be Only One...
Amazon's Silk browser, used in the Kindle Fire, implements SPDY and a reverse proxy cache in the Amazon cloud that is supposedly capable of predictive retrieval and caching. While it occasionally is faster than HTTP, on the whole it doesn't seem to mesh well with my browsing habits and I've disabled the so-called "accelerated page loading" on my KF. Judging from comments in the Amazon forums, my experience is not unusual.
The average is meaningless without the raw data. Suppose it averaged 5%: is that because all sites were 5% faster, or because one site was 500% faster and the others were 2% faster? The former would mean that SPDY is mostly useless. The latter would mean that SPDY is immensely useful, but just not in all cases.
You're not going to see the potential of SPDY before we have environments (browsers, CPU and your internet speed) that can take full advantage of it. Only in the most recent version of Firefox did we see SPDY support.
SPDY does not depend at all on CPUs or your "internet speed". It does depend on the browser (with both Firefox and Chrome supprting SPDY now) and, critically, the server. That last is also why the article author did not see much of a speedup - most content providers don't support SPDY yet. Going to non-SPDY servers and believing that it will evaluate SPDY for you is absolutely ridiculous.
Web browsing experiences are slowing down from advertising. But it's not an issue around the images that advertising loads. Instead, it is a combination of the extra time needed to load Javascript from advertisers (whether it is to spy on you or just to rotate ads around), and programming defects in that Javascript (doesn't play well with others). Browsers have to stop and wait for scripts to finish loading before allowing everything to run or even be rendered. You can have a page freeze in a blank state when some advertiser's Javascript request isn't connecting or loading.
The solution SHOULD be that browsers DISALLOW loading Javascript (or any script as the case may be) from more than one different hostname per page (e.g. the page's own hostname not being counted against the limit of one). This would remain flexible enough to reference scripts from a different server, or even an advertising provider, without allowing it to get excessive. Browsers should also limit the time needed to load other scripts, though this may be complicated for scripts calling functions in other scripts. Perhaps the rule should be that even within a page, scripts are not allowed to call scripts loaded from a different hostname, except the scripts from the page's hostname itself can call scripts in one.
I've also noted quite many web sites that just get totally stuck and refuse to even render anything at all when they can't finish a connection for some script URL. Site scripting programmers need to get a better handle on error conditions. The advertising companies seem to be using too many unqualified programmers.
now we need to go OSS in diesel cars
I came here to say something like "read the article, the guy is from Akamai and would know to only use servers that serve SPDY - such as many of the Google properties". But then, I read the fine article (blog) and realized the guy doesn't know enough to do that and just used "the top 500 sites" - which means a very large chunk of them don't know what SPDY is and he only used it between himself and his proxy. Great test that was. So your point is well taken. Bogus test means bogus results.
I used a Chrome browser as a client, and proxied the sites through Cotendo to control whether SPDY is used. Note that all the tests – whether HTTP, HTTPS or SPDY – were proxied through Cotendo, to ensure we’re comparing apples to apples.
Since they all ran through the same proxy, it might be the limiting factor. We would need to see tests the bypass the proxy to determine if his results have any meaning beyond that specific proxy.
Likewise, you have to ask the question, do all of the 500 sites he tested support SPDY?
make imaginary.friends COUNT=100 VISIBLE=false
No, he proxied the sites through a SPDY proxy, but that's not really a good way to test. Most sites shard domains to improve performance. Unfortunately, that basically destroys SPDY's pipelining advantage. The author tried to compensate by doing simple sub-domain coalescing (which he admits significantly increased SPDY performance where it applied) but that's just too coarse of an approach, as sharding is rarely ever restricted to subdomains. He also doesn't understand how browser parsing and loading works, and specifically that script execution and resource loads can be deferred (which is another key to keeping SPDY's pipeline full).
Basically, his tests showed that SPDY isn't magic faster dust. You will need to modify your site a bit if you want to really see advantages from it. And, you may see a minor performance degradation on an HTTP site that's unoptimized or heavily optimized in a way that doesn't play well with SPDY. However, if your site is optimized correctly, SPDY is still a big win over HTTP/HTTPS.
And unless that proxy used SPDY to talk to the servers, or had them entirely cached, that means nothing.
This is not at all true. There were numerous stories about how Google worked with CDNs to ensure compatibility with OpenDNS. Here's one example from last year:
http://arstechnica.com/tech-policy/2011/08/opendns-and-google-working-with-cdns-on-dns-speedup/
As for SSL False Start, the problem was a handful of SSL terminators that violated the spec. Unfortunately, most of the manufacturers of those devices showed no interest in making a trivial fix to their implementations, and the few that did make the fix didn't bother to deploy it to existing customers. So, everyone had to suffer because 0.5% of SSL servers were broken:
http://arstechnica.com/business/2012/04/google-abandons-noble-experiment-to-make-ssl-less-painful/
The reason SPDY doesn't help much is that your average web site now requires connections to a dozen other domains (each with only one or two requests), meaning that consolidating the requests to the main domain into a single connection just isn't that beneficial.
dom
IMHO, there is a huge issue with this test. On one hand its author claims : "However, when testing real world sites I did not see any such gains. In fact, my tests showed SPDY is only marginally faster than HTTPS and is slower than HTTP." But on the other hand at bottom lines it states : "This means SPDY doesn’t make a material difference for page load times, and more specifically does not offset the price of switching to SSL." SSL ? Wait a minute doc ... there is something wrong with the flux capacitor !
In fact to "SPDY"-fy the sites, the guy clarifies the technique he used :
"For a proxy, I used the Cotendo CDN (recently acquired by Akamai). Cotendo was one of the early adopters of SPDY, has production-grade SPDY support and high performance servers. Cotendo was used in three modes – HTTP, HTTPS and SPDY (meaning HTTPS+SPDY)."
You get it ? No, not the advertisement for cotendo ... ... He did not tested SPDY at all but tested TLS+SPDY !!! WTF ?!?
Either the guy should upgrade all his article replacing SPDY with TLS+SPDY or he should trash it and make another one only with SPDY. Then we will start to discuss about some of his strange benchmark decision...
Sad that such a odd article get so much press here :(
Shouldn't we be working on adopting SCTP instead?
As someone who's job it is to work on things like this, there's a few things that must be pointed out.
- SPDY runs over SSL. There isn't an unencrypted version -- note that SPDY was in fact faster than HTTPS.
- Many of the tricks used today to speed up page delivery, such as domain sharding, actually hurt SPDY's performance. SPDY's main benefit is that it opens up a single TCP connection and channelizes requests for assets inside that connection. Forcing the browser to establish a lot of TCP connections defeats this entirely, and the overhead of spinning up an SSL connection is very high. (And again, it should be noted that SPDY *WAS* faster, even if just a little bit, than standard HTTPS.)
There are other features in SPDY that today remain largely untapped, such as a server hinting to a client that it knows it'll need some content ahead of time -- giving the client something to do while it'd normally be idle waiting for the server to respond while it's generating the HTML it requested. (Large DB query, or whatever.)
Web engineers are clever and a smart bunch. While it looks like there's not a lot of gain to rethinking HTTP 1.1 today, given the years of organic growth we've had and time spent optimizing an older protocol, as new technology comes along that take advantage of the new foundation, things this will change. Give it time.
To the folks complaining that this guy doesn't know what he's doing, uh, he's a Chief Product Architect at Akamai. Yes he does. The folks at Akamai know more about web delivery than just about anyone.
- Bunny
1. HTTP Pipeline support proved very difficult to implement reliably; so much so that Opera was the only major browser to turn it on. It can be enabled in Chrome and Firefox but expect glitches. By all accounts SPDY's framing structure is far easier to reliably implement.
2. WIth SPDY, it's not just the content that's compressed but the HTTP headers themselves. When you look at the size of a lot of URLs and cookies that get passed back and forth, that's not a insignificant amount of data. And since it's text, it compresses quite well.
3. SSL is required for SPDY because the capability is negotiated in a TLS extension. Many people would argue that if this gets more sites to use SSL by default, that's a Good Thing.
4. If you're running SPDY, the practice of "spreading" site content across multiple hostnames, which improves performance with normal HTTP sites, actually works against you, since the browser still has to open a new TCP connection for each hostname. This is an implementation issue more than an issue with the protocol itself; I expect web developers to adjust their sites accordingly once client adoption rates increase.
5. The biggest gains you can get from SPDY, which few have implemented, is the server push and hint capability; this allows the server to send an inline resource to a browser before the client knows it needs it (i.e. before HTML or CSS is processed by the browser).
But as someone else as pointed out, the author's test isn't really valid, as he didn't test directly to sites that support SPDY natively, he went through a proxy.
The website I work for is supporting SPDY, and the gains we've seen are pretty close to the ~20-25% benchmarks reported by others. As many have pointed out, this author's methodology is way broken. I'd recommend testing to sites that are known to support SPDY (the best-known are Google and Twitter), with the capability enabled and then disabled (You can set this in Firefox's about:config, Chrome requires a command line lauch with --use-spdy=false in order to do this, though).
HTTP 1.1 pipelining does not allow more than one command to be outstanding. HTTP allows you to re-use the connection for your next command, but does not allow multiple commands at the same time.
HTTP1.1 is actually quite horrible for modern networks/computers. Choosing between HTTP1.1 and 1.0 is choosing between a giant douche and a turd sandwich. UDP won't happen because of the lack of congestion avoidance, which should be left up to the OS, not the app.
The biggest issue with HTTP is it does not allow multiplexing communication without opening up more connection, which is hard on servers, firewalls, and messes with TCP congestion detection.
no attempt is offered at explaining why SPDY should be slower than plain ol' HTTP, only why it might not be faster
The author was testing SPDY over TLS, which has a significant connection startup overhead. That probably explains all the performance degradation relative to regular HTTP.
In fact, if SPDY support was ubiquitous tommorrow, I would be surprised to see SPDY+TLS used for third party ad serving for this very reason. SPDY+TLS isn't likely to be used for that without _major_ standards modifications to allow third party content to be transparently proxied through first party sites cookies and all.
> It's enabled by default in IE.
Citation please? It wasn't last I checked, precisely because of issues like the ones I describe.
> if such transparent proxies are the issue the
> appropriate response isn't to cover up the
> problem.
What is the appropriate response? To ship software that doesn't actually work right when the user tries to use it?
> as a transparent proxy should simply only forward
> the request to the server or serve from the cache.
That's the theory. In practice, many such proxies are buggy. The rest if your comment is all about how a proxy is supposed to work. I know all that. My point was that many do NOT work like that. They do all sorts of internal request coalescing and reordering logic that works OK when the client is not using pipelining but makes the proxy behave in totally bogus ways when pipelining is present.
Here's a simple example: some proxies use a threadpool and every time they get a request they hand it off to some thread to service; that thread does a request on to the destination server and when the response comes back hands the data back down the socket the original request came in on.
With no pipelining, this setup is fine. With pipelining, the proxy gets a bunch of requests in a row on a single socket, and hands them out to different threads. These threads then run in parallel and start handing back data in parallel, all on the same socket. It's not what a proxy is _supposed_ to do with a pipelined connection, but it's what some proxies _do_. Theory and reality just don't always agree...