Are Web Pages Getting Larger?

An anonymous reader asks: "I work for a large multinational in a remote part of world. Our connectivity to the outside world (the Internet as well as company communications) is all done via a single E1 line - that's 2Mbps. Thousands of users. The company keeps access pretty well screwed down for security reasons, and the fact that our link to the outside world costs almost $300K/year! Our growing problem is Internet traffic. While policing of non-business use is very active, Internet traffic continues to grow. I'm becoming convinced that one of our problems is that average web page size is growing. As more of the world enjoys broadband access, I think web developers have less reason to limit the size of their web pages. Large images, flash animations and other size-increasing content seem increasingly common. Am I right? Can anyone point to a recent study that would support my theory, and help me convince my management that we just plain need more bandwidth?"

Comments

Often, but not always

[dtfinch]

A website and all of its pages can be expected to grow over its lifetime, but a lot of newer sites are lot smaller than previous generations. The wide adoption of CSS, and all the user friendliness tech evangalism emphasizing simplicity over noise has been paying off those who listen. There are still a lot of sites, such as web forums, where the attitude seems to be to make have really complex themes with almost no CSS and let mod_gzip/deflate deal with the task of making it small.

The report you are looking for should be called...

[Alpha27]

"Duh! Here's more content"

With the broadband market now including a minimum of 25% of home users, and up to maybe 40%, though I haven't looked at those numbers in some time, would be a contributing factor to the fact that yes, web pages are getting bigger.

One way to see proof of this is using the wayback machine.
http://www.waybackmachine.org/

I took a quick sampling of the NYTimes homepage, and noticed that the number has increased by a few kilobytes per year, from 56K in 2001, to 67K in 2003, to 83K in 2005. That's not even counting images. They've added more ad banners since the old days. If you google search, I'm sure you will find stuff.

Ad banners have increased in size, and complexity over time. Streaming content, is another addition, as well as more services running over the network.

You probably have a number of contributing factors happening to your bandwidth, in addition to web pages.
- Unless you have an internal instant messenging environment, you may have many ppl chatting away on services having to use your bandwidth.
- Email for personal use. Jokes, funny attachments, and worms clogging up things.

Here are a couple of suggestions to try and improve traffic:
- block services that shouldn't be run at the office like streaming music content.
- block websites that you see can have an impact on traffic, that you believe users should not be visiting. ie: quicktime movies.
- block your daytrading slacker coworkers.
- block ad servers entirely! this should drastically improve your situation, and be the easiest to implement.
- switch to an internal instant messenging service, if you haven't done so already.
- disable unnecessary services.
- ensure that you have an internet policy that prohibits the users from using their work companies for personal use.
- cache often used content.

trend seems to be up, but lately...

[pizza_milkshake]

according to archive.org/waybackmachine:
html size (doesn't include images/dependencies)
    slashdot.org    yahoo.com    microsoft.com
1996    -        7k        11k
1997    -        9k        -
1998    23k        10k        20k
1999    35k        10k        20k
2000    36k        12k        17k
2001    41k        16k        21k
2002    39k        17k        28k
2003    39k        32k        31k
2004    51k        33k        38k
Today    19k        14k        22k

the trend has certainly been up, but lately big sites' main pages seem to be slimming down, due to CSS as well as a tendency to store style and javascript in separate file

hmmm

[way2trivial]

http://www.google.com/search?hl=en&lr=&safe=off&q= average+web+page+size

has some good results
http://www.sims.berkeley.edu/research/projects/how -much-info/
has info from 2000 and a link to the same info from 2003

specific internet 2000
http://www.sims.berkeley.edu/research/projects/how -much-info/internet.html
and 2003
http://www.sims.berkeley.edu/research/projects/how -much-info-2003/internet.htm

it's worth noting, these types of statistics can take a year or more to compile..

Designers don't care about optimizing.

[paulbiz]

Web developers (and programmers in general) don't care about optimizing anymore, they just want it to be done so they can get paid. Worrying about such trivial things as a few kbytes or making valid and accessible HTML is asking too much of them.

From a web-designer standpoint, a lot of size can be reduced without altering the content.

Are you serving up nicely formatted HTML with indentations? That's wasteful. Strip whitespace and carriage returns.

Are you using HTML comments? Why? Does the customer really need to see them? Do you need to waste that bandwidth? Delete them or use comments in your server-side scripting language of choice.

Are you using GIF's where PNG's would be smaller? Or PNG's where GIF's would be smaller?

Have you optmized your PNGs, JPEGs and GIFs? (I don't remember a GIF optimizer, but there are plenty of non-destructive ones).

A 50x50 JPEG preview of an item does not need embedded comments, thumbnails, or EXIF data.

If you must use animated GIF's, be sure they are optimized and not full-frame!

Are you using pictures of words, when actual stylized text could convey the same message?

Are you using inline JavaScript or CSS, rather than calling it from a cacheable external file?

Are you using PDF, Flash or Java when it's not ABSOLUTELY necessary?

From a user's standpoint, the best solution, short of getting more bandwith: use less bandwidth. Turn off image loading or use a text-based browser. Don't browse the web as much. If you have a choice of sites to use, use the one that is smallest. Use a proxy. blah blah.

Use MRTG

[Matt Perry]

Can anyone point to a recent study that would support my theory, and help me convince my management that we just plain need more bandwidth?

The only study you need is a report from MRTG. Configure it and have it start graphing your network utilization for your E1. After a week or two you'll have several pretty graphs that can show your management exactly how saturated your connection is. Also, look at installing a caching proxy, such as Squid.