Slashdot Mirror
How Much Bandwidth is Required to Aggregate Blogs?
Kevin Burton writes "Technorati recently published that they're seeing 900k new posts per day. PubSub says they're seeing 1.8M. With all these posts per day how much raw bandwidth is required? Due to innefficiencies in RSS aggregation protocols a little math is required to understand this problem." And more importantly, with millions of posts, what percentage of them have any real value, and how do busy people find that .001%?
Actually I had been using gzip for quite a long time on a server at home and a few months ago I suggested to my boss that we install it on a few of our servers just as an evaluation. After some persuasion he finally agreed to the idea - mainly due to the fact that our IT manager had run gzip successfully on his home server too.
It all was really good to start with. Gzip was better than our expectations - server cpu usage was low and bandwidth costs had been reduced. Generally my boss was happy with the switchover - I had even been nominated for a promotion at the end of the year. Unfortunately we encountered one problem. One day the entire server drive corrupted due to a bug in the gzip code.
My boss wasn't happy with me at all. I was called into his office and given a stern reprimand. A few days later I was looking for another job from home. A word of advice: don't use gzip as all it seems to do is cause problems with its buggy code.