How Much Bandwidth is Required to Aggregate Blogs?

Kevin Burton writes "Technorati recently published that they're seeing 900k new posts per day. PubSub says they're seeing 1.8M. With all these posts per day how much raw bandwidth is required? Due to innefficiencies in RSS aggregation protocols a little math is required to understand this problem." And more importantly, with millions of posts, what percentage of them have any real value, and how do busy people find that .001%?

Comments

Re:Slashdot?

[Propaganda13]

If you'd check out my blog, you could read about the blogs I've read today thus saving yourself a lot of time.

Answer: Not much

[Anonymous Coward]

The bandwidth savings from using html+css are hugely exaggerated.

Slashdot is switching to html+css for the front page, but not for any dynamic pages like the one you're on now. Because slashcode was written by totally incompetent programmers, the markup for comment pages is not separated from the logic. Making any changes is therefore a huge undertaking and the people who wrote it are far too busy maintaining the high journalistic standards slashdot is known for to do it.

Wheat from chaff

[StikyPad]

search query: blog -1337 -teh -kewl -hugz -omg -bored -lol -lmao -"can't wait to get my drivers license"

Re:Wheat from chaff

[Rosco P. Coltrane]

search query: blog -1337 -teh -kewl -hugz -omg -bored -lol -lmao -"can't wait to get my drivers license"

Ah! I guess you missed the following blog entry then:

Hi everybody, it's Sunday today and I'm bored. So I guess I'll get on with my homemade engine that runs on water. As you know, it's almost finished, and I expect it to put out as much as 1337 horsepower. The reliability of the motor should be good too: my friend, Ray Kewl in engineering, said it should provide well beyond 10,000 TEH (total engine hours).

Update: the engine is in the car, and it runs! on nothing but water! OMG I'm so happy! check the pictures and the diagrams to build your own. I can't wait to get my drivers license renewed so I can take it for a spin!

Re:How much? If everyone GZipped, a lot less!

[TooncesTheCat]

God you really are trying to argue semantics on a fucking moot point.

I'm too tired to explain to you how retarded that comment is in context to a multi-million dollar business like a datacenter. You think that they care if you are using 30 more Watts of electricity which doesnt equate to them having an extra 100 dollars on their power bill. They dont care / would never raise rates because of their power bill....They only raise rates when bandwidth availability / rackspace becomes a premium or their demand goes up. Not just because of something as trivial as your 30 extra Watts of power being used because your using Gzip.

And your acting like Gzip would be maxing your CPU out 99% of the time.

People that argue semantics piss me off.

Re:Bandwidth wasted for non-xhtml pages?

[oh_bugger]

A more serious question is how much bandwidth /. is wasting by hosting the large quantity of duped articles

Re:How much? If everyone GZipped, a lot less!

[tsm_sf]

yah, I see that:

  1) you're trying to have a conversation about two separate topics w/ 2 separate people
  2) you've mixed up both the topics and the people already
  3) you've replied to your OWN posts when you meant to reply to someone else's
  4) you really like the word 'semantics'

Have to say that I'm really enjoying the fact that you work in IT but get pissed off by ppl arguing over linguistics. The irony is maxing my CPU out.