How Much Are Ad Servers Slowing the Web?

vipermac writes "Most of the times I have a problem with a Web page loading slow or freezing temporarily, I look down at the status bar and see that it's waiting on an ad server, Google Analytics, or the like. It seems to me that on popular Web sites the bottleneck is overwhelmingly on the ad servers now and not on the servers of the site itself. In my opinion we need a better model for serving ads — or else these services need to add more servers/bandwidth. Are there any studies on the delay that 3rd-party ad servers are introducing, or any new models that are being introduced to serve ads?"

Comments

use firefox and adblocker!

[fred fleenblat]

problem solved.

Re:use firefox and adblocker!

[spyder913]

There are still ads on the internet? I sometimes forget, until I have to open up IE.

Re:use firefox and adblocker!

[Maniac-X]

Or if you like ads (sometimes the google ones are amusing, or you want to support the website you're visiting), turn on HTTP Pipelining. It'll handle all of your requests simultaneously instead of one after the other.

Re:use firefox and adblocker!

[Variorum]

But if you use AdBlock your a thief! At least according to this /. article http://slashdot.org/article.pl?sid=07/08/17/135920 6

Re:use firefox and adblocker!

[X0563511]

Well then, feel free to call me a rampant kleptomaniac.

Re:use firefox and adblocker!

[X0563511]

I thought pipelining was when you took one connection and used it for downloading multiple resources, saving bandwith and time on connection re-establishments.

I see what you are getting at though. There is no reason that you should load a websites resources in serial if they span multiple hosts. Not sure why web browsers do that. (making sure that there is only one transfer to/fro a single host makes it easier for the servers to manage their load - lower load over more time rather than spikes of large load)

Re:use firefox and adblocker!

[skoaldipper]

Pipelining? So Ted Stevens was right!

By the way, I set pipelining.tube.maxrequests to 128. The googles, they do nothing...

Re:use firefox and adblocker!

[Anonymous Coward]

Kleptomaniac? Is the the KDE p2p app?

Re:use firefox and adblocker!

[PitaBred]

They are worth it, though. The mouth-breathing morons who can't block ads like we do are the types to click on the ads or punch the monkey. I consider annoying ads a "tax" for people who can't be bothered to educate themselves marginally about the exceptionally complex system they want information from.

Re:use firefox and adblocker!

[Kadin2048]

You mean some browsers serialize that? My browser appears to fetch multiple hosts concurrently.... Unfortunately, it still won't render the page with certain missing content (e.g. slow-loading javascript crap from ad servers). Well, the RFCs only allow 2 connections per client to one webserver; any more is considered abusive. And to support some older webservers, most browsers only get one page element per connection. So they open a connection, send one GET request, let the server respond, close the connection ... rinse, repeat.

This was OK on narrowband/dialup connections (in fact, most browsers used to render the page between elements by default, so that it would show you the whole page, then re-render as various images or other elements were downloaded and ready to display -- although as connections got faster relative to rendering time, most browsers switched to only rendering the page once when it was complete), but it sucks on broadband. As the amount of time each data transfer takes drops relative to the time required to establish the connection, the establishment and resetting of the connections for each page element becomes more "expensive."

So in HTTP/1.1 they introduced a way of making multiple requests in one connection. (It may have predated HTTP/1.1 but I think that was when it was first formalized). Basically the web browser opens a connection to the server and make multiple requests at once. Then the server will respond with all the requested elements. Then the connection will close. This is considered kosher and non-abusive because it doesn't require spawning a whole lot of connections at the server; everything is done in one.

However this isn't enabled in default in Firefox; you have to go into the about:config page and turn it on, and set the number of requests per connection to something reasonable (I think 8 is the max).

Also, it requires a certain amount of intelligence on the part of the browser to do this correctly. There are certain kinds of requests that shouldn't be pipelined (PUT requests, for instance), and some older servers may not like it. However, I think we're moving pretty quickly towards a time where it can be made the default.

Re:use firefox and adblocker!

[ttfkam]

So in HTTP/1.1 they introduced a way of making multiple requests in one connection. (It may have predated HTTP/1.1 but I think that was when it was first formalized). Close. It was possible to make multiple requests per connection in even the HTTP 1.0 spec. HTTP 1.1 simply made that behavior the default. In other words, you had to specify an additional header to support multiple requests in 1.0, but in 1.1 you had to specify "Connection: close" to prevent the behavior.

Good introductory overview in general. Kudos.

Re:use firefox and adblocker!

[ajs318]

Quicker way:

$ sudo echo "127.0.0.1 www.block.this.site" >> /etc/hosts

No need to muck about with vi. (If you do want to advise people to use vi, it's good form then to tell them the exact keystrokes they will need. Some people aren't as smart or brave as the secretaries who used to work for AT&T, and do actually get put off by a screenful of tildes, a beep every time they press a key and no obvious way out. So: shift+G to Go to the end of the file; a to enter append mode; type the extra line; ESC to get out of {or escape from} append mode; shift+Z, shift+Z to save and exit {go to sleep ZZ}.)

Re:use firefox and adblocker!

[v1]

But isn't that like saying if you don't look at the bilboard as you drive down the highway, or at the ad poster on the subway, or walk into the kitchen for a snack during the commercial break, that you're a theif? The television example is probably the best and easiest to associate with.

Advertisements are an opportunity to make an impression on a customer. While there will always be technologies that make it difficult to ignore the ad, in most cases you are not obligated to be impressioned. Just because the advertisers get upset that you are stealing the cheese from their trap, it's your prerogative.

I'm a little surprised that we don't see more "banner" ads on TV. Imagine all these people with the widescreen sets that are viewing content with black sides because it's 4x3 formatted instead of widescreen. Imagine banners on both of those dead zones on the sides, that change every 20-30 seconds and adjust their product to something related to the main feature. Y know I think I would prefer that to the "four miniutes of commercials every 10 minutes" we get now. Also, even though a lot of shows are timed for like 49 minutes for the hour to accomodate commercials, a lot of stations trim out scenes or cut them short to insert more ads, so we would be getting more content. I wouldn't mind them doing this so long as they were not animated. Sort of like how I can't stand the animated shockwave banners here and at other sites. I don't understand why no one is doing this already. Though I have seen a few isolated examples of banner ads being shot across the bottom of the screen briefly during some shows - those are overlays though and degrade the content so maybe that's why those have been unsuccessful.

Sheesh

[Verteiron]

"Nothing for you to see here. Move along."

Must be 'cause I'm using Firefox...

Like slashdot?

[Anonymous Coward]

I probably would have had first post if slashdot did not serve up so many ads!!

Jokes aside, I do notice waiting for ads on slashdot quite often but it is one of the few sites that I allow more to get through.

Browser's fault?

[DoofusOfDeath]

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

I realize this means performing some speculative page layout that may need to be re-done when the dimensions of the ads are served. But it sure would beat waiting tens of seconds to see the page's real content.

Re:Browser's fault?

[ePhil_One]

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

I've always assumed this was intentional, to buy the add more "eyeballs", or awareness, or whatever. Server teh add, wait 3 seconds, load the rest of the page. Make sure they get a good look at that ad...

Re:Browser's fault?

[Anonymous Coward]

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

It depends how the ad is served. If it's served as an external piece of JavaScript (using a script element), then most browsers will reach the script tag and won't render anything else until the script has been downloaded. This can cause a delay if the ad server is slow or down.

If the ad is served using an img, iframe or object element, you generally don't have this problem, as the browser can leave a space for the advert and carry on rendering the rest of the page.

I work for an ad serving company and most of the ads we serve are in iframe elements. The growing popularity of script elements (they seem to be used for most third-party ads now) confounds me. Generally, I'm continually surprised at how much control over the user experience most websites are willing to give to ad serving companies.

Re:Browser's fault?

[pedramnavid]

I work for an ad serving company and

release the hounds.

Re:Browser's fault?

[dpu]

Another option is to use the "DEFER" option in the script tag. Any script within the tags will wait until the page loads before executing. I wish ad companies would start using that *sigh*

Re:Browser's fault?

[Shados]

They don't want to, obviously, because you may end up going away from the page (cuz you realised it wasnt the right one) before the ad loads, unfortunately.

Re:Browser's fault?

[FireFury03]

Is it possible for browsers to render everything *else* on a page while awaiting the ads to be served?

Most ad systems seem to work by placing a <script> tag where you want the ad to appear which loads a script from the ad server that does a document.write() to insert the actual code. This is very bad practice (and explicitly disallowed for XHTML) but even Google do it (which sucks since I have to jump through all sorts of hoops to get AdSense to work on my XHTML site).

document.write() works by actually writing out HTML and feeding it into the parser and thus parsing the page must be suspended at that point until it's finished executing, so you can't render the page until the advert has loaded.

The _correct_ way to do this is for the ad-serving Javascript to actually modify the DOM tree. But that requires the ad server developers to not be lazy and have clue, which seems to be asking too much. (or alternatively, don't use Javascript at all).

Re:Browser's fault?

[ZachPruckowski]

I just want to briefly say that "screenful of code" and "Stupidly easy" are antonyms, not synonyms, in this day and age. Since many sites are run by people for whom HTML is a challenge and Javascript latin (people who install Wordpress/Movable Type/whatever), these sorts of problems go unsolved.

I'm not flaming bloggers by saying (or at least not intentionally). What I mean is that the bar for web publication has been lowered (and by and large it's a good thing, too) so that anyone with more than basic computer skills can get a blog. On the whole, this is great for the web, the Marketplace of Ideas, and society at large, but it does result in problems like this. Specifically, website creators delegate the idea of working out the details of the ads to the ad-provider, and just copy-paste in the ad code. Admittedly, if I had a blog (and I may get one soon), I'd probably start out doing the same thing until I felt more comfortable with the HTML/CSS/JS required.

Re:Browser's fault?

[jollyreaper]

>> I work for an ad serving company and

> release the hounds.

Jesus, is your last name Vick? I try to be careful about what my dogs eat.

Re:Browser's fault?

[Shados]

Good point. When I posted that, I had in mind the typical mainstream sites, like news web sites, this very one, etc. Web sites made by professionals. Even blog web sites tend to have a few software engineers behind em, the bloggers don't make the engine.

Don't see THAT many ad driven web site made in MS Word these days...

0 slowdown for me

[jandrese]

For the reasons mentioned in the op I have several notorious slow adservers in my /etc/hosts. I don't know if they're still a problem, but doubleclick used to be horrible about taking 10 or 15 seconds to get their ad bits back to you. I'm not even particularly zealous about killing ads, but if you're stalling out my webpage then it's in /etc/hosts for you.

Re:0 slowdown for me

[halcyon1234]

If you're interested in populating your hosts file, check out http://www.mvps.org/winhelp2002/hosts.htm. There's a downloadable hosts file that's 406k, and was updated on July 31st, 2007. If you're running 2000/XP/Vista, be sure to read the Editor's Note about steps you must take to use a large hosts file.

Re:0 slowdown for me

[LiquidCoooled]

Additional to my note, looking through the specific host file you reference I see it blocks a load of porn sites.

These are wanted and are not crappy adverts

ahem..

Re:0 slowdown for me

[Carnildo]

I use localhost, which means my private webserver returns a 404 right quick.

Re:0 slowdown for me

[halcyon1234]

Those could very well be porn sites that'll deliver more than just pictures of naked people to your computer.

But hey, if you enjoy virus-laden sex, then I'm sure you'll find lots of dirty, dirty places to send your header requests to...

Re:0 slowdown for me

[LiquidCoooled]

Viruses mainly come from insecure browsers and user stupidity.

The biggest (solvable) problem with porn (and other) sites is link/statusbar hijacking where you click one link and are redirected to another site not related.

The solution involves preventing access to the statusbar and unfolding the links (with a greasemonkey script).

I will not click anything which I cannot see the resultant page.

Re:0 slowdown for me

[thomas.galvin]

I'm not even particularly zealous about killing ads, but if you're stalling out my webpage then it's in /etc/hosts for you.

Agreed. For the most part, I'm happy to let ads load, but there are four things that will get you added to my killfile real quick:

* Boobies (or anything else that might get me fired)
* Shaking, flashing, screen grabbing, "look at me!" type stuff
* Fake dialogs and error messages
* Slowdown

I just started filtering for the last one recently. For about a week straight I kept seeing "waiting for ads.doubleclick.so.goram.slow" in my status bar, and eventually I just adblocked their entire domain.

Also, slashcode is apparently unable to figure out what to do with ul and li tags.

no slowdown at all...

[Endymion]

...for me at least. Blocking Google Analytics, Doubleclick, etc, with noscript has made my browsing experience much smoother. Not only is it nice to not have the random pauses while it hits the ad-server, not running the javascript has helped the render time on some pages as well (even if you still run the javascript for the page itself!)

Display the page before the data's all loaded

[mh101]

Yes, I had noticed it recently too, where the page isn't displaying because of waiting for a response from an ad server.

So why don't all web browsers start displaying the data they do have, rather than waiting for the ad server to submit it's data first? If there's a delay in downloading an image on the site or a style sheet it still starts displaying and when the image/stylesheet is downloaded the page is re-rendered to reflect that. So what is it about the page design that forces web browsers to not display anything if the delay is due to an ad server?

Re:Display the page before the data's all loaded

[ZachPruckowski]

Javascript is usually involved. Because Javascript is single-threaded and does in-order execution, if an ad uses Javascript, then waiting on that javascript to finish will hold up the rest of the page.

Re:Display the page before the data's all loaded

[cerberusss]

Oh, JavaScript is multithreaded alright. I'm currently working for a fortune 500 company and I'm using a technique involving 35 pop-ups where popup 1 is a message-passing hub for all other pop-ups. Together these form the heart of our business of racks and racks full of dual Xeons pumping out ads like there's no tomorrow.

I would make it cross-browser too, if it weren't for F^(&(* AdBlock.

High-CPU Flash Ads

[Kenshin]

What pisses me off are badly designed Flash ads. They use plenty of CPU power just to animate something completely useless. Last year Dell was running this ad on my local newspaper's site that took 80% of my CPU just to animate FALLING SNOWFLAKES. I complained to the website, and they took it down.

Some Flash ads barely take any CPU at all, and those are honestly fine by me, but some just hog my resources. The problem is that the people who DESIGN these ads typically have cutting-edge machines, so they don't know what it's like to run them on a shitty office machine. So, please, TEST your ads on a shitbox average computer before you force them on us!

Re:High-CPU Flash Ads

[StarfishOne]

I know all too well what you're describing. Quite a bit of relief is found in the following Firefox extension:

http://flashblock.mozdev.org/

This extension replaces flash movies/objects with a button you can click to view them. You can also whitelist certain websites (e.g. YouTube) to always show the flash movie directly.

I do have to note though that when trying to whitelist a website I am viewing at that very moment, the extension does crash my browser every now and then. It might be something on my machine though! Anyway, with FireFox being able to restore sessions now, I find it a 'must have' extension to block out those CPU-consuming flash movies. I truly hate them

Agreed

[garett_spencley]

I run a few web sites and on some I have a geo-IP targeted ad that loads in an iframe. This particular ad is often a bottleneck so I wanted to solve it. My first idea was to run a wget on the server and cache the output to the hard disk so I can load the ad from the server instead of a 3rd party. This would also require one less DNS look-up.

Then I realized that it would completely fail because the ad is geo-IP. So the cache will always display the location of my server, and not the user.

The obvious solution is for ad companies to offer scripts to their affiliates that could be run on the servers hosting the sites. Of course that opens up new problems, like security issues. But if the code were open we could spot such issues.

In fact, that seems to me like such a simple and obvious solution. The only reason that ad companies don't do that (that I can think of) is that they want to appeal to people running on free hosts where they can't run server-side scripts. But there's no reason not to offer both IMO. I also thought that they wanted to keep things as absolutely simple as possible, and there's nothing simpler than saying "just copy/paste this into your html document". But any web master who rents hosting (shared or dedicated) knows how to upload a php script.

what ads?

[ianare]

What ads are you guys talking about, I see barely any at all. *turns off ad block plus, refreshes* Holy crap! How do you even go online like this? You might as well just watch TV.

Re:what ads?

[tachyonflow]

Ha! This is funny *and* insightful. A co-worker and I are frequently researching web sites, checking out the potential competition to our business ideas, and such. More often than not, I look at a web site and say, "I totally don't understand their business model. How can they fund this sort of a web site without any ads at all? Oh... wait..." Many times it's: "XYZ web site has ads now, you say? No way! When did they start putting ads up?" Heh...

Re:what ads?

[tuxedobob]

I'm at the point where I just don't see images on a page. I don't block them, or use any add-on software. I just don't look at them. When I'm going to a web site, I'm looking for text, and the text is all I look at.

As soon as there's an ad that covers the text I want to look at, just close the window. I don't care who gets paid, but no one's getting any of my money, and the web site obviously wasn't that important if it was trying to cover up the content.

Abusers aren't satisfied with one kind of abuse.

[Futurepower(R)]

The people who make ads are a self-destructive bunch. Numerous times I've waited for a Slashdot page to load while some ad server took its time. Abusing me with abusive, dishonest ads wasn't enough, they wanted to abuse me by wasting my time, too. Mentioning the problem to Slashdot editors brought only temporary fixes, or no change.

So now I don't see the ads at all, thanks to Firefox's AdBlock Plus and NoScript add-ons. (I recommend NoScript only for people who don't mind fiddling with permissions for each new web site.)

I guess abusers aren't satisfied with only one kind of abuse. I can dimly remember some of the Slashdot ads. When they weren't misleading, they were generally stupidly written. People with no technical knowledge shouldn't work for technical companies.

Ads? or Webmasters?

[Evets]

The big problem is that most webmasters design their sites in such a way that they are dependent on a third party product being available prior to their pages being rendered.

Google isn't always up. Plenty of times, I see issues because my comcast connection can't see the google servers even though everyone else can get to them just fine.

It's entirely feasible to write your page in such a way that it can display data before any other files are loaded. Serve up ads in an iframe, include tracking images in an iframe or as the last element of a page, etc.

But ads aren't the only thing causing page load problems. Third party widgets, crazy fat CSS and JS files, and pages with way too many images are still a problem.

That is the reason I use a filter

[SmallFurryCreature]

Back in ages long gone, when firefox did not exist you had (still have perhaps) a company called doubleclick whose adservers would sometimes choke freezing the loading of the rest of the page. Why and how this happens? Do I look like someone who gives a shit?

I wanted it gone, and finally I bit the bullet and read up on squid and available plugins and setup my linux router to just filter all http traffic. Haven't looked back since.

Browsing without a blocker is like... well it just sucks. At times I am offcourse forced to browse the web without such blocking software and my god, the internet has become as bad as tv. Do they really think that if you saturate people with advertising to the point the original content becomes unusable people are really going to be more inclined to buy?

Apparently so. However not to me. This story offcourse neatly links to the story below about a site block firefox because of adblocker.

Well, who gives a shit. You went to far, now you gotta pay the price. If you don't get revenue from me, blame doubleclick and all those others who just pushed me over the edge.

At the moment I recommend bfilter to people who are fed up as well, it is browser neutral, works out of the box and does a lot more then just ad-blocking. Granted some flash bits require you to click them before they actually load but that is okay, because 99% of flash stuff I don't want to load.

So yes, ad-servers are slowing the net, by adding stuff to webpages I do not want. Can this be solved? It has been solved, not to the liking of those who depend on those ads being seen, but hey, fuck them. Do they care when I have to reload a page over and over again because some server borked?

Re:Use Adblock with my subscription...

[blueturffan]

I took one look at your website and immediately clicked away. No offense intended, but it didn't look like a site I would trust downloading anything from.

Blocked because I'm paying for the pipe

[schwit1]

My internet connection ain't free. If the ad folks want to use MY bandwidth they should pay me for the privilege.

Re:Blocked because I'm paying for the pipe

[dballanc]

"My internet connection ain't free. If the folks want to use MY bandwidth they should pay me for the privilege."

It's funny that both the user and the website owner share the same argument. As usual in life, it's the few who make it rough for the many. Most people don't mind reasonable ads, and they don't mind contributing financially to a site they enjoy. Unfortunately all it takes is a few greedy jackass types taking the ads to extremes and ruin the concept entirely.

It's too bad there isn't a advertising standard that sites can be certified with and filters can be aware of. An ad whitelisting service, that legitimate companies would value as much as a BBB or google page ranking.

Re:Blocked because I'm paying for the pipe

[Anonymous Coward]

If the ad folks want to use MY bandwidth they should pay me for the privilege.

How about they give you free content instead of paying in cash?

Re:Blocked because I'm paying for the pipe

[munrom]

Welcome to Australia, I've got 42GB cap and that's one of the larger ones

Re:Progressive loading

[SpankTheUser]

I work for a Very Large Online Publisher, and yeah, this is pretty much the key. There are few reasons for an ad or traffic counter to slow a properly designed page. In fact, displaying pages that render regardless of ads can turn out to be mission critical. A year ago, a story on one of our sites got Drudged, Slashdotted, Boing-Boinged, Dugg, etc. all at once. This is not totally unusual for us, and should not have been a problem for us to handle. However, the servers really started to slow down, and while the sites never went down completely, surfing our content got pretty painful. Culprit turned out to be an interesting interaction between users, ad servers, and our site. We had a slow loading ad that was appearing on most of our page. Because the HTML wasn't well designed, the page content wouldn't load until the ad appeared. Users presumably have figured this out, and would click re-load on their browser over and over again until the ad server finally responded to one of their requests. Result: actual number of requests to our servers grew by an order of magnitude over what we'd normally expect for a given number of actual users. We wrote up a presentation entitled "Slow ads + bad HTML = Company Left $XXX,XXX On the Table" and got the funding to re-code all our templates.

How about just fewer ads?

[Mr. Fahrenheit]

In my opinion we need a better model for serving ads -- or else these services need to add more servers/bandwidth.

I know that this doesn't speak specifically to the rest of your question, but IMHO, we need a better model than having ads. Just because we can have 'em doesn't mean we should all the time. It seems to me that the click-throughs, browser-tracking, etc., benefit the ad companies themselves far more than the individual content providers.

I realize I'm tilting at windmills here, but the current web ad-model has even city and local community web pages (like libraries) littering their pages with 'ads' for other parts of the same site, etc. It is really quite annoying.

/...and stay off my lawn!

Turning on pipelining

[claykarmel]

This was greek to me. Here's how.

Turn it on this way:
http://www.mozilla.org/support/firefox/tips

And information about how to access the secret tools (Why didn't I know this until now? I must be lame.)
http://www.mozilla.org/support/firefox/edit#aboutc onfig

Re:ignorance, selfishness and jerkiness

[White Flame]

so that way websites will be forced to go on pay to see/subscription models. what a fantastic way to go. from open, free internet to newspaper stand format.

"open, free internet" is what we had before the ads, tracking mechanisms, malicious exploits coming through said ads, and other privacy invasions existed.

"onlythievesblockthem"

[Leperous]

Re. the logic that it's "stealing" to block ads: "1% of people seeing our adverts will buy something from it" does not mean that "1% of adverts seen by any given user result in a sale." Simply put, some people will not ever buy stuff from advertising, ever, and if anything, it's stealing from them. I see no problem in blocking ads if they're paying the host on a per-click basis, rather than per-view.

Re:use firefox and adblocker! (mod up parent)

[Teun]

At least you get mod points. I haven't had the ability for easily five years now despite consistently having excellent karma. Interesting, how does an AC get karma?

Privoxy, Adblocker & NoScript

[avicarmi]

Privoxy can also be installed on windows, it is browser independent.