Slashdot Mirror

I, for one, welcome our new overlords on Ruby on Rails on Mac OS X (roromoxes?).

Especialy when I hope to be one of them :D.

In what (will be) new in rails 1.1, you will find more interesting stuff about that beautiful framework and its future. Worth reading if you are flirting with RoR.

for well over a year http://scottraymond.net/archive/4745#4745

this has already been the topic of much discussion

Here is my original long-winded essay which explicated the RSS+BT idea:

http://scottraymond.net/archive/4745

After addressing the initial whys and wherefores, I speculate on how the pairing might be potent enought to spark an indie media revolution. Here's the text:

-- RSS meets BitTorrent meets TiVo.

The other day, Steve Gillmor wrote about BitTorrent and RSS and how they could be combined to create a "disruptive revolution." He's half right. RSS and BT are indeed two great tastes that taste great together, but Gillmor's vision is upside down: we shouldn't use BitTorrent to carry RSS, we should use RSS to carry BitTorrent. Let me explain.

-- But first, some background.

RSS (RDF Site Summary) is a simple format for syndicating content on the web. These days, the most common application of RSS is subscribing to weblogs: you tell your computer to check an RSS file for changes every so often, and then it notifies you when there's something new to read. If you're like me and you read one metric shitload of news every day, this is a life-saver.

BitTorrent, the brainchild of Bram Cohen, is the current cool-kids' P2P program. It works sort of like Kazaa, but at a lower level. It doesn't handle searching for new files, it doesn't have a media player, it just concentrates on downloading big files efficiently.

Okay. Two solutions in search of a problem. Here's a problem:

-- I have a weakness.

I am addicted to the show Alias. I watched the first couple episodes of season two as it aired, and I was hooked. In my honest moments, I'll admit that the show's appeal is mostly due to the callipygian Jennifer Garner. It's a weakness; we deal.

But it gets worse. I go out on Sunday nights, when Alias airs, and I don't want to give that up. That's why God created the VCR, I know, but to compound the problem, I don't have TV. I don't want to have TV, because I love the feeling of superiority that I get by not having it.

This system is at tension, it has no rest, its forces are unbalanced, it wants to be resolved.

-- A partial answer.

The internet, it turns out, is great at resolving different kinds of tensions, and this is one of them. After a few weeks of missed episodes, I realized that with a little patience, a P2P program like Kazaa was able to fetch back-episodes with aplomb. Each file is around 450 megs, fairly high-quality video, with commercials cut out. I start a few episodes downloading, and by the next evening, they're ready to watch, whenever I have the time.

After a few weeks of enjoying this, a new tension emerged: I had caught up with all of the old episodes, and I had to wait a week for each new one. The problem is that the Kazaa protocol isn't especially well-tuned for getting brand new files: first someone has to record the show as it airs, cut out the commercials, and compress it to a reasonable size, then seed it on the network. Then, it has to slowly propagate to its peers, each transfer taking hours. It might take three days before it's available on enough peers that I'm able to even find it, let alone download it.

-- BitTorrent to the rescue.

The solution is BitTorrent. BitTorrent operates on similar principles to Kazaa, but it's tuned differently: it excels at downloading files that are new or currently in high demand. It breaks large files into many small chunks, and coordinates their assemblage, so that users can tap into a swarm and distribute the load evenly. At the same time that you're downloading a chunk, another user is downloading an earlier chunk from you -- no one server is overwhelmed, and the more popular a file, the higher its availability is. It's perfect for large files that are most interesting when they're fresh -- in other words, it's perfect for TV shows.

In many cases, I have been able to use BitTorrent to completely download a new TV show mere hours after the show airs.

Sounds like an application for broadcatching .

Creating a torrent for each patch would ensure the fastest possible distribution, while the rss feed would ensure the timely announcement of it.

"broadpatching" ?

I was thinking along similar lines for distributing virus signature updates. What with "warhol worms" now routinely spreading faster than traditionally-downloaded (and slashdotted signature files, antivirus vendors need a more efficient distribution channel, as well.

Here's a gentle introduction to the BT/RSS concept that I wrote in December:

(from http://scottraymond.net/archive/4745)

- RSS meets BitTorrent meets TiVo.

Steve Gillmor wrote about BitTorrent and RSS and how they could be combined to create a "disruptive revolution." He's half right. RSS and BT are indeed two great tastes that taste great together, but Gillmor's vision is upside down: we shouldn't use BitTorrent to carry RSS, we should use RSS to carry BitTorrent. Let me explain.
But first, some background.

RSS (RDF Site Summary) is a simple format for syndicating content on the web. These days, the most common application of RSS is subscribing to weblogs: you tell your computer to check an RSS file for changes every so often, and then it notifies you when there's something new to read. If you're like me and you read one metric shitload of news every day, this is a life-saver.

BitTorrent, the brainchild of Bram Cohen, is the current cool-kids' P2P program. It works sort of like Kazaa, but at a lower level. It doesn't handle searching for new files, it doesn't have a media player, it just concentrates on downloading big files efficiently.

Okay. Two solutions in search of a problem. Here's a problem:

- I have a weakness.

I am addicted to the show Alias. I watched the first couple episodes of season two as it aired, and I was hooked. In my honest moments, I'll admit that the show's appeal is mostly due to the callipygian Jennifer Garner. It's a weakness; we deal.

But it gets worse. I go out on Sunday nights, when Alias airs, and I don't want to give that up. That's why God created the VCR, I know, but to compound the problem, I don't have TV. I don't want to have TV, because I love the feeling of superiority that I get by not having it.

This system is at tension, it has no rest, its forces are unbalanced, it wants to be resolved.
A partial answer.

The internet, it turns out, is great at resolving different kinds of tensions, and this is one of them. After a few weeks of missed episodes, I realized that with a little patience, a P2P program like Kazaa was able to fetch back-episodes with aplomb. Each file is around 450 megs, fairly high-quality video, with commercials cut out. I start a few episodes downloading, and by the next evening, they're ready to watch, whenever I have the time.

After a few weeks of enjoying this, a new tension emerged: I had caught up with all of the old episodes, and I had to wait a week for each new one. The problem is that the Kazaa protocol isn't especially well-tuned for getting brand new files: first someone has to record the show as it airs, cut out the commercials, and compress it to a reasonable size, then seed it on the network. Then, it has to slowly propagate to its peers, each transfer taking hours. It might take three days before it's available on enough peers that I'm able to even find it, let alone download it.
BitTorrent to the rescue.

The solution is BitTorrent. BitTorrent operates on similar principles to Kazaa, but it's tuned differently: it excels at downloading files that are new or currently in high demand. It breaks large files into many small chunks, and coordinates their assemblage, so that users can tap into a swarm and distribute the load evenly. At the same time that you're downloading a chunk, another user is downloading an earlier chunk from you -- no one server is overwhelmed, and the more popular a file, the higher its availability is. It's perfect for large files that are most interesting when they're fresh -- in other words, it's perfect for TV shows.

In many cases, I have been able to use BitTorrent to completely download a new TV show mere hours after the show airs. Like a TiVo user, I'm no longer bound to a specific time to watch my shows. I'm free to go out on Sunday night and still watch my show while it's brand new. TV is now asynchronous.

- Life is good.

But it could be bet

The article proposes using BT to transport RSS files, with the goal of reducing the load on extremely popular RSS files. It's a fine idea, except that RSS files are generally very small, and BT incurs such overhead that it's a poor choice for distributing small files.

The real potential of combining BT & RSS is in the reverse: use RSS to distribute .torrent files, or .torrent URLs. Configure your news aggregator to pass all new .torrents to the BT client, and you've got yourself a media aggregator that uses open standards, and is easy on the content provider's servers.

That combination allows indy media producers (audioblogs, videoblogs, etc) to reach a mass audience in a scaleable way, without overwhelming bandwidth costs. And, it allows media consumers to subscribe to feeds from trusted sources -- be they official of unofficial -- instead of hunting for the .torrent of the latest Alias episode. Or whatever.

Here's my more fleshed-out response to Gillmor's article, on my weblog.