When RSS Traffic Looks Like a DDoS

← Back to Stories (view on slashdot.org)

When RSS Traffic Looks Like a DDoS

Posted by ryuzaki0 on Tuesday July 20, 2004 @07:32AM from the which-is-most-of-the-time dept.

An anonymous reader writes "Infoworld's CTO Chad Dickerson says he has a love/hate relationship with RSS. He loves the changes to his information production and consumption, but he hates the behavior of some RSS feed readers. Every hour, Infoworld "sees a massive surge of RSS newsreader activity" that "has all the characteristics of a distributed DoS attack." So many requests in such a short period of time are creating scaling issues. " We've seen similiar problems over the years. RSS (or as it should be called, "Speedfeed") is such a useful thing, it's unfortunate that it's ultimately just very stupid.

7 of 443 comments (clear)

Over the years? How about over the weekend? by Marxist+Hacker+42 · 2004-07-20 07:37 · Score: 5, Informative

We've seen similiar problems over the years. RSS (or as it should be called, "Speedfeed") is such a useful thing, it's unfortunate that it's ultimately just very stupid.

And it seems to have gotten worse since the new code was installed- I get 503 errors at the top of every hour now on slashdot.

--
SJW: a person who perceives an injustice, and while correcting it, commits a greater injustice.
Re:Simple HTTP Solution by skraps · 2004-07-20 07:45 · Score: 5, Insightful
This "optimization" will not have any long-lasting benefits. There are at least three variables in this equation:
1. Number of users
2. Number of RSS feeds
3. Size of each request
This optimization only addresses #3, which is the least likely to grow as time goes on.
--
Karma: -2147483648 (Mostly affected by integer overflow)
Re:What about a scheduler? by cmdr_beeftaco · 2004-07-20 07:50 · Score: 5, Funny

Bad idea. Everyone knows that most headlines are made at the top of the hour. Thus, A.M. radio always give news headlines "at-the-top-of-hour." RSS reader should be given the same timely updates.
Related to this is the fact that most traffic accidents happen "on the twenties." Human nature is a curious and seemingly very predictable thing.
Re:Idea by ganhawk · 2004-07-20 07:53 · Score: 5, Interesting

You could have a system based on JXTA. Instead of the bittorrent model, it would be something like the P2P Radio. When the user asks for feed, a neigbour who just recived it can give it to the user (overlay network, JXTA based) or the server can point to one of the users who just received it.(similar to bittorrent but user gets whole file from peer intead of parts. The user also does not come back to server at all, if transfer is successfull. But the problem is this user need not serve others and can just leech)

I feel overlay netwrok scheme would work better than Bittorrent/tracker based system. In overlay network scheme each group of network will have its own ultra peer (JXTA rendezvous) which acts as tracker for all files in that network. I wanted to do this for slashdot effect (p2pbridge.sf.net) but somehow the project has been delayed for long.

--
Python script to convert photos into "artsy" portraits: http://p2pbridge.sf.net/pyPortrait/
Oh, come on by aiken_d · 2004-07-20 07:54 · Score: 5, Interesting

My guess is that InfoWorld is dynamically generating the RSS for each request. A simple host-side cache of the generated XML, so hits just talk to the HTTP server and not the app server, would probably make this a non-issue.

Or are they *really* getting more RSS hits than image requests? If -- somehow -- that's the case, spend $500/mo on Akamai or Speedera and point RSS stuff there, and give the CDN a reasonable timeout (30 minutes or something). That guarantees you no more than about 500 hits per timeout period, or maybe one every 10 seconds. Surely the app server can handle that.

Then again, what do I know? I only worked there for five years, including two on infoworld.com. It's been a few years, but unless things have changed dramatically, that is one messed up IT organization.

Cheers
-b

--
If I wanted a sig I would have filled in that stupid box.
Re:RSS needs better TCP stacks by Salamander · 2004-07-20 07:58 · Score: 5, Insightful

Leaving thousands upon thousands of connections open on the server is a terrible idea no matter how well-implemented the TCP stack is. The real solution is to use some sort of distributed mirroring facility so everyone could connect to a nearby copy of the feed and spread the load. The even better solution would be to distribute asynchronous update notifications as well as data, because polling always sucks. Each client would then get a message saying "xxx has updated, please fetch a copy from your nearest mirror" only when the content changes, providing darn near optimal network efficiency.

--
Slashdot - News for Herds. Stuff that Splatters.
RSS is like a DDoS attack on my brain by PCM2 · 2004-07-20 08:05 · Score: 5, Interesting

Am I the only one who finds it easier to get the information I want from the home pages of the sites I trust, rather than relying on an RSS feed? For one thing, in an RSS feed every story has the same priority ... stories keep coming in and I have no idea which ones are "bigger" than others. Sites like News.com, on the other hand, follow the newspaper's example of printing the headlines for the more important stories bigger. With RSS, it's just information overload, especially with the same stories duplicated at different sources, etc. Everyone seems really excited about RSS, but when I tried it I just couldn't figure out how to use it such that it would actually give me some real value vs. the resources I already have.

--
Breakfast served all day!