Slashdot Mirror
RSS And BitTorrent, Together At Last
eoyount writes "Wired has an interesting story about a really simple idea I wish I had thought of. Transferring large files across the Internet isn't easy for your average joe, but a combination of RSS and BitTorrent technology might just make it easier - Slashdot ran a previous story on the theoretical blending last year." (LegalTorrents is run by the strangely familiar simoniker, who wrote a short piece on the O'Reilly Network about how it was set up, and offers observations on how well the combination fares.) Update: 03/17 21:45 GMT by T : Ernest Miller submits two related postings he's written on RSS+BitTorrent, a combination he calls "broadcatching."
I've been writing extensively on Corante about RSS + BitTorrent, which I call "Broadcatching" here: Broadcatching Archives See, for example, RSS + BitTorrent Roundup - Broadcatching Isn't MS Active Channels and First Broadcatching App Available! (And Related News).
BitTorrent is basically another p2p service, except it's different (yes, i'm trying to be very specific here)
It allows for people to take advantage of bandwith by downloding bits of a large file from different users hosting a 'torrent.' At the end, all these pieces are put together. Yes, it is pretty good.
...combining RSS and torrents is not going to solve the problem. This is the most complicated solution to a non-problem that I've seen since someone paid me to design something.
where can I find clear info regarding what is RSS exactly ? Isn't it somehow like what microsoft tried to do a couple years ago with their "Active Desktop" (c) TM concept ? Or am I completely way off ?
Je n'ai pas d'avenir Je n'ai qu'un destin Celui de n'être qu'un souvenir C'est pour demain
Now, how do we aim that at the web in general (and automagically) to avoid the slashdot effect?
A new browser protocol? Aim your browser at
bthttp://www.victim.com
and let it rip?
Although it's cool that companies are finding legit uses for BT (I believe the Worlds of Warcraft beta is being distributed this way), I'm not sure the legal departments are up to speed yet. To quote one of the fellows in my IRC chat:
"Hrm, WoW is bing distributed by Bittorrent. Meanwhile, I get angry phonecalls from Vivendi to shut down Bittorrent."
Yay for technical advances, but can commercial interests fully embrace it without killing the "evils" of it?
Kids these days. They don't know the difference between classic, and just plain old.
The problem with bittorrent is that a lot of users disconnect as soon as their download is finished. Won't this be an even bigger problem with game downloads (specifically multiplayer games) since even if the users knows they should stay connected afterwards, they might not since it would lag their game?
Many ISPs and college campuses block P2P ports, BitTorrent included. I'm not sure that 'news' is a compelling enough reason to have many (or any) of them change their policies.
Trolling is a art,
This sounds like exactly what SuprNova.org needs. It would relieve some of the server load on their main pages and would enable them to serve more .torrent files.
-molo
Using your sig line to advertise for friends is lame.
People keep trying to make BitTorrent something it isn't. And really, we should be fighting its corporate adoption in any form, as it's simply an attempt to shift server bandwidth costs to the client. ISPs eat that right now, but we're going to metered access if this keeps up.
Which is effectively getting us to pay for website access/services, but instead of giving the money to the content creators we'll be giving it to ISPs instead and paying in bandwidth besides. So this is a bad idea.
In this Slashdot article, Yahoo reported that things might be starting to come together. Looks like it's happened!
However, I'm a little concerned - BitTorrent has a lot of initial overhead (setting up trackers, and all the protocol stuff). I'm not sure if it would be wise for small files?
This sounds like a wonderful melding of two current technologies...
However, remember when cable gained enough steam to warrant not one but many 24hr cable news networks? We are now blessed with an overabundance of crappy sensationalist "reporting". I do NOT want cnn/msnbc/fuxnews/etc. landing on my HD.
If an individual set up a feed for say, a favorite game or movie alone, I would subscribe. But most webpages I read, I gloss over quickly then am done with.
If I, and everyone else had subscribtions to all of the media content of their favorite websites delivered autonomously, the majority of it getting thrown out quickly...
think of the bandwith, the poor helpless bandwith, won't somebody please think of the child., er bandwidth!?
...but in that case, you're no worse off than before. And realistically, if there's some huge download I'll usually just start it before I head to bed. Of course, if you're sitting there counting down the seconds until it's done, that's different...
Kjella
Live today, because you never know what tomorrow brings
Nice attempt at a troll, but by using BitTorrent at all, you are uploading. (Either that, or you want us to believe that you use a hacked client that will download at a screaming rate of 0.1k per second, max.) P.S. You take a greater risk of death every time you step in a car than you run a risk of being sued when you upload a file in Kazaa.
BitTorrent's weak spot has always been thedistribution of the torrent files in the first place. If there isn't a torrent file on the conent provider's page, where do you look?
RSS+BitTorrent, is a step closer to a better web. It almost answers the problem of pointing your client at an actively downloaded torrent by steering users twoard a slimmer and more flexible protocol.
IMO, maybe some kind of 'standard' torrent directory/lookup that is guarnteed to be traded by all torrent clients is the right ticket; kind of like a DNS for media. The RSS+Torrent scheme is good, but all it does is displace the complexity of the matter onto a new protocol and rely on everyone hitting the same feed to begin (the problem Torrent is trying to eliminate).
It does however, make it easy to make distributing torrents a lot more dynamic. Neat stuff.
Isn't this just Konspire2b? Konspire2b was designed specifically for this purpose:
Konspire2b
Essentially you subscribe to channels which push content instead of pulling.
Compared to Bittorrent
This is an exhaustive analysis (with pretty charts) why under the above scenario (pushing content, as opposed to pulling), Konspire2b is much more efficient.
It's 10 PM. Do you know if you're un-American?
Please provide proof that BitTorrent is 'based' on KaZaA sourcecode. Of course, you can't.
:)
:)
I do know that Bram talked with the authors of Furthur, an open-source JAVA P2P for legal content. A few members of the Furthur dev team also work for a company that once did buisness with Sharman Networks, so if anything, KaZaA source may be based on already GPL-ed software... but don't tell anyone that
Of course, all modern P2P is 'based' on Napster or Gnutella (take your pick), so it's all a mute point anyway
does BitTorrent even work under Linux?
Would you like the GUI client or the command line one?
Yes, it works very nicely under Linux.
"Ignorance more frequently begets confidence than does knowledge"
- Charles Darwin
The previous poster is incorrect. BitTorrent has nothing to do with Kazaa (0 lines of code in common).
BitTorrent is open source (MIT license) and written in Python.
Kazaa is closed source, spyware-ridden dreck that was probably written in C++.
Bittorrent protocol has nothing in common with the protocol used by Kazaa (FastTrack). Even their basic P2P topologies are different.
Also, Kazaa is in trouble not for it's protocol, but for running servers that allow piracy, it's just in Kazaa's case one automatically means the other, since the protocol is closed source. Of course, Bittorrent trackers that host pirated material are also susceptable to such troubles - but this has nothing to do with Bittorrent protocol itself.
Uh, yes...
Here, here, and here.???
Sorry if I seem like I'm trolling but these questions will be asked at some point
-Phil
Shoot questions, first ask later...
The best thing about this idea is that it plays right into the strength of bittorrent - namely, having a large number of people trying to get the same content at the same time. Since everyone will get the RSS feed at roughly the same time, there will be a large number of peers to share the load for bittorrent.
The funny thing is, I ran into Andrew the other day, and he was just gushing about this new idea he had! I had no idea what he was talking about at the time. Guess I missed my chance to post a story on slashdot.
On stereophonic equipment, the monaural sound obtained through multiple channels will enhance your listening pleasure.
Yep, that's the point of BitTorrent - and what made it so special when it first came out
So the sharing still works to some extent even if people disconnect right away.
Furthermore, it's only possible to leech significantly from a peer that has already finished downloading. Any other client will notice that it's giving but not getting, and scale down it's giving appropriately. I bet the leechs love it when folks moralise about not disconnecting right away - on the other hand, it's certainly not the kiss of BitTorrent when those connections do drop.
right now we've got a 90's-style ol' skool web interface for distributing our media, teamed up with ol' skool ftp/http mirrors distributing things around the globe.
... we are prime users for free, open, public, easy-to-use media sharing technologies, and if the big-guns aren't using it, we share are happy to!
but, we'll definitely use an rss-fronted bittorrent network, if and when it can actually be smoothly integrated with our existing setup.
ampfea stands for 'a meeting place for electronic artists' and its a community-supported media hosting/community service for a bunch of muso's
check out some of our files sometime. its all home-made music...
; -- the corruption of government starts with its secrets. a truly free people keep no secrets. --
This site has been offering RSS feed of links to BitTorrent animes for couple of months. Firefox and RSS reader extension are great for pulling down all BitTorrent links.
One step closer to Nirvana. Bittorrent + RSS + gentoo emerge --update world. Or how about something that uses software-suspend to automagically hot-swap in the latest bleeding edge kernel? Maybe the Hurd allows on-the-fly kernel upgrades.
Since data sent equals data recieved within a BT swarm, and some people will act as seeders and continue to send more data than they recieved, you will always have people who will simply not have the opportunity to contribute to the swarm, mostly at the tail end. And of course many folks have their uploads limited or even completely cut off.
The real problem with bittorrent is that by enabling efficient transfer of large files, people are transferring larger files. And the service providers simply do not have the capacity for everyone to be sending those large files. They may advertise unlimited access but kids they really aren't set up for it. To say nothing of the fact that the way the internet is structured now is no longer geared towards everyone being as able to send as well as they are to recieve.
Really, the internet and its billing structure should be geared towards billing by amount received, and not amount served, and widespread implementation of load-sharing protocols like bittorrent. It would be far more efficient and fair, and would encourage people to limit their consumption rather than penalizing inadvertently popular unsupported sites.
---If you can't trust a nerd, who can you trust?
This problem is easily addressed with multicasting. All a server need do is send a multicast datagram to notify all RSS syndicators that the RSS document has been updated, at which time the syndicators can fetch the new document.
It may seem complicated but its not really.
The main problem people have using bittorrent is regressive internet connections. (Until IPv6 becomes ubiquitous this is going to be a problem for many of the internet's designed uses, not just swarming media.)
Im not so hot about RSS, but for things such a multicast or bittorrent- it really helps to get the content when everyone else is. So having a running subscription to a show you like, then have the download automatically kick-in as soon as it becomes available would be the ideal setup for using your computer as a media center.
Getting this working right could make even tivo seem quaint...
True multicast could help, too. But seeing as cable companies cleverly bought the routers you use (unless you're in academia or the military or both), they're just NOT going to turn on multicast routing any time soon, unless more people are aware of the possibilities and start leaning on them HARD.
In the right hand corner: Hackers Embrace P2P
"I could wake up in the morning and find the latest recordings from my favorite band loaded into my portable MP3 player, and just pick it up and go."
I don't understand where they are coming from here. If I am going to pay to download music, which consists of relatively small files, I am not going to run a BT to help out an online music store.
When they start mentioning uses so far off the base of reality, the whole article starts to smell of BS. Especially since the slowest part of the MP3 experience tends to be copying music from the PC to the player.
Typically, I load new tracks on my ipod before leaving in the morning. I'll tag the stuff, then transfer it before I hop into the shower. As far as downloading goes, I can download a whole CD of music in ~10 minutes. The only way the article's method would be worth doing would be if you invested in huge libraries of online music purchases daily.
On that note: Please quit looking to solve problems that don't exist.
BT is non-linear as you suggest. The n'th person gets the n'th chunk. This still allows for (randomly-caused) relative scarcity of certain chunks (although they are not the last ones!), and that is the problem you notice up around 98%.
No, seriously, try playing a partially complete BT download of an AVI with a player that doesn't look for the index (mplayer, DivX, etc.). The file is missing random chunks, not the end.
The previous sig has been removed due to
A million years ago (1998?) Wired published a whole edition on Push as the Next Big Thing. It was the first time I was really aware of them being totally wrong. Or perhaps just a bit ahead of their time.
While I think this is a neater solution, there is another product that does exactly the same thing, allow you to subscribe to channels and received pushed content via incentive compatible (you get faster speeds if you upload more) swarms.
It's called kast.
The way I figure it, with this bittorrent-RSS combination and a slight modification of torrent watching sites like animesuki we will essentially have a fansubbed anime online tivo at our disposal. Actually, you could have probably done that even without RSS, though it does simplify matters. The only limitations are our bandwidth and hard drives. Which actually are pretty limiting these days, especially with p2p being frequently capped.
Hell, you could modify an actual TiVo with broadband for exactly this sort of thing, and it needn't be limited solely to anime either. I'm sure it'll be popular with overseas watchers of American TV as well.
The international media and internet companies need to face facts and realize that Video On Demand is a reality and is already extremely popular - but that the shows people are demanding are not the ones the companies have been providing through their own limited, misfocused, and (most importantly) redundant services. Until we see simultaneous worldwide release of all media (including DVDs released simultaneously with the theatrical release) they will find themselves losing what should have been their easiest sales - those to impatiently eager fans.
---If you can't trust a nerd, who can you trust?
Realistically speaking, the biggest problem with Bittorrent is seeding. I think this is how bittorrent works:
.torrent file generated .torrent file is uploaded to a tracker .torrent from the tracker .torrent file, which causes the the bittorrent client asks the tracker for the machines/locations of the seeds and people downloading the file(s) pointed to by the .torrent
.torrent goes missing, the file is inaccessible. If the tracker goes away, the file is inaccessible.
* a file is seeded, and a
* that
* clients who want to download the file download the
* the user opens the
* the client downloads various chunks of the files from both the seeds and the other downloaders
The more people download a file, the better bittorrent is able to spread the bandwidth.
The downside is that if a file isn't seeded, it's no longer available. If a
Bittorrent's main problem right now, which is a client problem, is its upstream usage can easily swamp a home connection. That's just dumb client design.
Upload limiting works, but limits your download speed. The client develoeprs have to recognize that yes, sharing is nice and leeching is bad, but disrupting the users' connection is a Very Bad Thing.
It's a shame they're using RSS, as it's a good idea with a bad implementation. There are currently 9 different versions of RSS, and all of them incompatible with one another. It ought to be replaced with a better technology like Atom. However, this does look like an interesting project, nonetheless.
-- Rob
Y'a jamais des choses qu'on peut pas se débrouiller ; juste laisse-moi t'aider!
Think of BitTorrent as an updated version of ftp that utilizes the clients upload bandwidth as well as the servers download bandwidth. That's all it is. (Yes, clients become servers even before the download process is complete.)
If the author of BitTorrent could be sued the authors of ftp and apache could be sued as well.
I hope this helps, your misconception of how BT functions is fairly common. It's not the same type of p2p network as Kazaa at all.
Encryption. It should generate a unique key pair for each socket it opens.
If I take a knife and I cut someone up with it. Does that mean that somehow you are now going to be unable to go use the knife for something you want/need to?
Honestly, if the only thing on this planet that anyone used bittorrent for was "warez", *GASP* you could still set up torrents for legal files and have all your buddies download them.
If BT becomes illegal at some point, then we all are going to have a lot more to worry about then the fact that we can't get ahold of our warez.
./revolution
Here's a gentle introduction to the BT/RSS concept that I wrote in December:
(from http://scottraymond.net/archive/4745)
- RSS meets BitTorrent meets TiVo.
Steve Gillmor wrote about BitTorrent and RSS and how they could be combined to create a "disruptive revolution." He's half right. RSS and BT are indeed two great tastes that taste great together, but Gillmor's vision is upside down: we shouldn't use BitTorrent to carry RSS, we should use RSS to carry BitTorrent. Let me explain.
But first, some background.
RSS (RDF Site Summary) is a simple format for syndicating content on the web. These days, the most common application of RSS is subscribing to weblogs: you tell your computer to check an RSS file for changes every so often, and then it notifies you when there's something new to read. If you're like me and you read one metric shitload of news every day, this is a life-saver.
BitTorrent, the brainchild of Bram Cohen, is the current cool-kids' P2P program. It works sort of like Kazaa, but at a lower level. It doesn't handle searching for new files, it doesn't have a media player, it just concentrates on downloading big files efficiently.
Okay. Two solutions in search of a problem. Here's a problem:
- I have a weakness.
I am addicted to the show Alias. I watched the first couple episodes of season two as it aired, and I was hooked. In my honest moments, I'll admit that the show's appeal is mostly due to the callipygian Jennifer Garner. It's a weakness; we deal.
But it gets worse. I go out on Sunday nights, when Alias airs, and I don't want to give that up. That's why God created the VCR, I know, but to compound the problem, I don't have TV. I don't want to have TV, because I love the feeling of superiority that I get by not having it.
This system is at tension, it has no rest, its forces are unbalanced, it wants to be resolved.
A partial answer.
The internet, it turns out, is great at resolving different kinds of tensions, and this is one of them. After a few weeks of missed episodes, I realized that with a little patience, a P2P program like Kazaa was able to fetch back-episodes with aplomb. Each file is around 450 megs, fairly high-quality video, with commercials cut out. I start a few episodes downloading, and by the next evening, they're ready to watch, whenever I have the time.
After a few weeks of enjoying this, a new tension emerged: I had caught up with all of the old episodes, and I had to wait a week for each new one. The problem is that the Kazaa protocol isn't especially well-tuned for getting brand new files: first someone has to record the show as it airs, cut out the commercials, and compress it to a reasonable size, then seed it on the network. Then, it has to slowly propagate to its peers, each transfer taking hours. It might take three days before it's available on enough peers that I'm able to even find it, let alone download it.
BitTorrent to the rescue.
The solution is BitTorrent. BitTorrent operates on similar principles to Kazaa, but it's tuned differently: it excels at downloading files that are new or currently in high demand. It breaks large files into many small chunks, and coordinates their assemblage, so that users can tap into a swarm and distribute the load evenly. At the same time that you're downloading a chunk, another user is downloading an earlier chunk from you -- no one server is overwhelmed, and the more popular a file, the higher its availability is. It's perfect for large files that are most interesting when they're fresh -- in other words, it's perfect for TV shows.
In many cases, I have been able to use BitTorrent to completely download a new TV show mere hours after the show airs. Like a TiVo user, I'm no longer bound to a specific time to watch my shows. I'm free to go out on Sunday night and still watch my show while it's brand new. TV is now asynchronous.
- Life is good.
But it could be bet