Slashdot Mirror
Are Long URLs Wasting Bandwidth?
Ryan McAdams writes "Popular websites, such as Facebook, are wasting as much as 75MBit/sec of bandwidth due to excessively long URLs. According to a recent article over at O3 Magazine, they took a typical Facebook home page, looked at the traffic statistics from compete.com, and figured out the bandwidth savings if Facebook switched from using URL paths which, in some cases, run over 150 characters in length, to shorter ones. It looks at the impact on service providers, with the wasted bandwidth used by the subsequent GET requests for these excessively long URLs. Facebook is just one example; many other sites have similar problems, as well as CMS products such as Word Press. It's an interesting approach to web optimization for high traffic sites."
compression to shorten the URL's?
Wordpress by default allows you to configure URL writing. The default is set to something like: http://www.mysite.com/?p=1.
For SEO purposes it's always handy to switch to the more popular example: http://www.mysite.com/2009/03/my-title-of-my-post.html.
Suggesting that we cut URL's that help Google rank our pages higher is preposterous.
Are forums (fora?) like these wasting bandwidth as well by allowing nerds, like myself, to banter about minutia (not implying this topic)? Discuss amongst yourselves.
Read the rest of this comment
Absolute power corrupts absolutely. indymedia
The PHPulse framework is a great example of a better way to do it. It uses one variable sent for all pages which it then sends to a database (rather than an XML page) where it stores the metadata of how all the pages interelate. As such, it doesn't need to parse strings, it is easier to build SEO optimized pages and it can increase page load times by 10 times over other MVC frameworks.
This is my sig. There are many like it but this one is mine.
The short Facebook URLs waste bandwidth too ;)
Tsunami -- You can't bring a good wave down!
By default Wordpress produces short urls.
Good is never enough, when you dream of being great!
Of all things that could be optimized, urls shouldn't have a high priority (unless you want people to enter them manually).
I'm pretty sure their HTML, CSS, and javascript could be optimized way more than just their urls.
But rather than simply sites, people often what it to be filled with crap (which nobody but themselves care about).
ps, that doesn't mean you should try to create "nice" urls instead of incomprehensible url that contain things like article.pl?sid=09/03/27/2017250
It's irrelevantly small portion of the traffic, while at the scale of Facebook, it could save some traffic, but does not make any impact on the bottomline worthwhile the effort!
150 chars long url = 150 bytes VS 50KILObytes + Images of rest of the pageview....
I'm throwing out of my head that 50kilobytes for the full page text, but a pageview often runs at over 100kb.
So it's totally irrelevant if they can shave off the 100kb a whopping 150bytes.
Pulsed Media Seedboxes
Twitter clients (including the default web interface) auto-tinyURL every URL put into it. Clicking on the link involves not one but *2* HTTP GETs and one extra roundtrip.
How long before tinyurl (and bit.ly, ti.ny, wht.evr...) are cached across the internet, just like DNS?
This is ridiculous. If I have a billion dollars, I'm not going to worry about saving 50 cents on a cup of coffee. The bandwidth used by these urls is probably completely insignificant.
---Technology will liberate us if it doesn't enslave us first.
75 whole freaking megabits? WOWSERS!!!!
They must be doing gigabits for images, then. Complaining about the URLs is complaining about the 2 watts your wall-wart uses when idle, all the while using a 2kW air conditioner.
Oh, you're not stuck, you're just unable to let go of the onion rings.
This is a stupid exercise. Oh my gosh, there's an extra few characters wasted. They're talking about 150 characters, which would be 150 bytes, or (gasp) 0.150KB.
10 times the bandwidth could be saved by removing a 1.5KB image from the destination page, or doing a little added compression to the rest of the images. The same can be said for sending out the page itself gzipped.
We did this exercise at my old work. We had relatively small pages. 10 pictures per page, roughly 300x300, a logo, and a very few layout images. We saved a fortune in bandwidth by compressing the pictures just a very little bit more. Not a lot. Just enough to make a difference.
Consider taking 100,000,000 hits in a day. Bringing a 15KB image to 14KB would be .... wait for it .... 100GB per day saved in transfers.
The same can be said for conserving the size of the page itself. Badly written pages (and oh are there a lot of them out there) not only take up more bandwidth because they have a lot of crap code in them, but they also tend to take longer to render.
I took one huge badly written page, stripped out the crap content (like, do you need a font tag on every word?), cleaned up the table structure (this was pre-CSS), and the page loaded much faster. That wasn't just the bandwidth savings, that was a lot of overhead on the browser where it didn't have to parse all the extra crap in it.
I know they're talking about the inbound bandwidth (relative to the server), which is usually less than 10% of the traffic. Most of the bandwidth is wasted in the outbound bandwidth. That's all anyone really cares about. Server farms only look at outbound bandwidth, because that's always the higher number, and the driving factor of their 95th percentile. Home users all care about their download bandwidth, because that's what sucks up the most for them. Well, unless they're running P2P software. I know I was a rare (but not unique) exception, where I was frequently sending original graphics in huge formats, and ISO's to and from work.
Serious? Seriousness is well above my pay grade.
Seriously. Long URL's as wasters of bandwidth? There's a flash animation ad running at the moment (unless you're an ad-blocking anti-capitalist), and I would expect it uses as much bandwidth when I move my mouse past it as a hundred long URL's.
I'm not apologizing for bandwidth hogs... back in the dialup days (which are still in effect in many situations), I was a proud "member" of the Bandwidth Conservation Society, dutifully reducing my .jpgs instead of just changing the Height/Width tags. My "Wallpaper Heaven" website (RIP) pushed small tiling backgrounds over massive multi-megabyte images. But even then, I don't think a 150-character URL would have appeared on their threat radar.
It's a drop in the bucket. There are plenty of things wrong with 150-character URLs, but bandwidth usage isn't one of them.
Stressed? Me? Of course not. Stress is what a rubber band feels before it breaks, silly.
For an even more egregious example of web design / CMS fail, take a look at the HTML on this page.
$ wc wtf.html
12480 9590 166629 wtf.html
I'm not puzzled by the fact that it took 166 kilobytes of HTML to write 50 kilobytes of text. That's actually not too bad. What takes it from bloated into WTF-land is the fact that that page is 12,480 lines long. Moreover...
$ vi wtf.html
Attention Globe and Mail web designers: When your idiot print newspaper editor tells you to make liberal use of whitespace, this is not what he had in mind!
In order to maximize the web experience for all customers, effective immediately all websites with URLs in excess of 16 characters will be bandwidth throttled.
Sincerely,
Comcast
Are YOU using the TOOL, or is the TOOL using YOU? Think about it!
Has anyone here even looked at what the real motivation behind this study is? It's to create this idea that web hosts, are, surprisingly, wasting the valuable bandwidth provided by your friendly ISPs. Do a few stories like this over a few years, and suddenly, having Comcast charge Google for the right to appear on Comcast somehow seems fair. The bottom line is, as a consumer, its my bandwidth and I can do whatever I want with it. If I want to go to a web site that has 20,000 character URLS, then, that's where I'm headed.
This is my sig.
I hope this is obvious to most people here, but reading some comments, I'm not sure, so...
The issue is that a typical Facebook page has 150 links on it. If you can shorten *each* of those URLs in the HTML by 100 characters, that's almost 15KB you knocked off the size of that one page. Not huge, but add that up over a visit, and for each visit, and it really does add up.
I've been paying very close attention to URL length on all of my sites for years, for just this reason.
Just use a smaller font for the URL!
Actually, when I had my web page designed (going on 4 years ago), I specifically asked that all of the pages load in less than 10 seconds on a 56k dialup connection. That was a pretty tall order back then, but it's a good standard to try and hit. It's somewhat more critical now that there are more mobile devices accessing the web, and the vast majority of the country won't even get a sniff at 3G speeds for more than a decade. There is very little that can be added to a page with all the fancy programming we can put into them. Mostly, I (and my clients who need to find me) want information, and one of the best ways is simply readable text with static pictures. For the web, you can really compress the heck out of an image and still have it look crisp on a monitor.
Is it just my observation, or are there way too many stupid people in the world?
ebay has "upgraded" their local site http://my.ebay.com.au/> and "my ebay" is now a 1M byte download. That's ONE MILLION BYTES to show about 7K of text and about 20 x 2Kb thumbnails.
The best bit is that the htm file itself over 1/2 Mbytes. Then there's two 150K+ js files and a 150k+ css file.
Web "designers" should be forced to develop on a 128M P3 machine with VGA screen and dial up modem
-- Butlerian Jihad NOW!
Interestingly (or maybe not), Google doesn't gzip their analytics javascript file...
Are Long URLs Wasting Bandwidth?
No. But this article is.
Isn't Facebook itself the huge waste of bandwidth as opposed to just the verbose URLs it generates?
Mind the gap...
...except they aren't using mod_gzip/deflate. At first I thought you browsed the web RMS style and maybe wc* didn't support compression** and you were just getting what you deserved***, but then I checked in firefox and lo and behold:
Response Headers - http://www.theglobeandmail.com/blogs/wgtgameblog0301/
Date: Fri, 27 Mar 2009 23:39:54 GMT
Server: Apache
P3P: policyref="http://www.theglobeandmail.com/w3c/p3p.xml", CP="CAO DSP COR CURa ADMa DEVa TAIa PSAa PSDa CONi OUR NOR IND PHY ONL UNI COM NAV INT DEM STA PRE"
Keep-Alive: timeout=15, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html
200 OK
No compression!
If they had been using compression, it would have made all that whitespace fairly negligible.
Probably a result of how their template system stitches everything together. Still, that is pretty bad. There is no excuse to run a webserver and not turn on compression. It is the single biggest way to boost page-load and decrease bandwidth.
* wget 4 lyfe!
** compression is probably evil and Anti-Freedom(tm) somehow, kinda like images are evil or fads like "graphical user interfaces" are evil. In otherwords,anything that makes things easier or faster for a user is basically evil and Anti-Freedom****.
*** braindead comment spamming bots are the only thing not using compression (except RMS, probably)
**** I'll leave it to you, dear reader, to deduce if I'm serious. Hint: no hint.
...the first 1831 lines (!) of the page are blank...Attention Globe and Mail web designers: When your idiot print newspaper editor tells you to make liberal use of whitespace, this is not what he had in mind!
Believe it or not, someone had it in mind. This is most likely a really, really stupid attempt at security by obscurity.
PHB:My kid was showing me something on our website, and then he just clicked some buttons and the entire source code was available for him to look at. You need to do something about that. ::whispering to WebGuy #2:: Just add a bunch of empty lines. When the boss looks at it, he won't think to scroll down much before he gives up.
WebGuy:You mean the html code? Well, that actually does need to get transferred. You see, the browser does the display transformation on the client's computer...
PHB:The source code is out intellectual property!
WebGuy:Fine. We'll handle it.
PHB:Ah, I see that when I try to look at the source it now shows up blank! Good work!
The problem isn't bandwidth, it is that long URLs are a pain from a usability standpoint. They cause problems in any context where they are spelled out in plain text (instead of being hidden as a link). For example, they often get broken in two when sent in plain text email. When posting a URL into a simple forum that only accepts text (no markup), a long URL can blow-out the width of the page.
Where does this problem come from? It comes from SEO. Website operators realized that Google and other search engines were taking URLs into account, so CMSs and websites switched from using simple URLs (like a numeric document ID) to stuffing whole article titles into the URL to try to boost search rankings. One of the results of this is that when someone finds a typo in an article title and fixes it, the CMS either creates a duplicate page with a slightly different URL, or the URL with the typo ends up giving a 404 error and breaks any links that point to it.
What I don't understand is why search engines bother to look at anything beyond the domain name when determining how to rank search results. How often do you see anything useful in the URL that isn't also in the <title> tag or in a <h1> tag? If search engines would stop using URLs as a factor in ranking pages, people would use URLs that were efficient and useful instead of filling them with junk. The whole thing reminds me of <meta> keyword tags -- to the extent that users don't often look at URLs while search engines do, website operators have an opportunity to manipulate the search engines by stuffing them with junk.
Have you tried compiling the whitespace =)
The HTTP-Referer isn't designed for ?ref=somesource
Your stat software wants to know if more people click to your page through the logo ?ref=mylogo or through a link in the story ?ref=story. The Referer can't give you that info.
The HTTP-Referer also is no good for aggregation. It only give you a URL. If you didn't append something like ?campaign=longurl, it would be almost impossible to track things like ad-campaigns.
HTTP-Referers *are* good for dealing with myspace image leeches. If you haven't I suggest you read thorough you log files right now--I bet you'll find 20% of your traffic is myspace idiots leeching your images. Redirect those guys to something more... tasteful, and enjoy the bandwidth savings.
Wow. Judging by the patterns that I see in the "empty" lines, it looks like their CMS tool has a bug in it that is causing some sections to overwrite (or in this case, append instead).
I'd bet that every time they change their template, they are adding another set of "empty" lines here and there, rather than replacing them.
-David
This is silly. The URLs, even "long" ones are miniscule compared to the pictures, streaming video, music, javascript etc. on these pages. To worry about them is like worrying about the lint on a suit of clothes making them too hot. This is just absurd.
To fthr sav our bdwdth, ^A txt shld be cmprsd into acrnyms!