Slashdot Mirror
Interview Responses From BitTorrent's Bram Cohen
1) Bit-Torrent browsing... by CashCarSTAR
Has any effort/thought been put towards bit torrent page distribution?
Specifically, a way that one can use BT to mirror webpages. A way to get around the /. effect, and as well would work wonders the next emergency that comes out (see 9/11).
Bram:
Images in web pages are very small and require very low latency. BitTorrent is designed for much larger files, which download on the order of minutes or hours rather than seconds. BitTorrent uses the significant amount of time those downloads take to try out and compare different connections. This process has inherent latencies which make it unsuitable for images on web pages.
Certainly it would beis possible on paper to dramatically reduce the cost of hosting an ordinary web site using peer transfers, but the logistical problems of handling many small files at low latency have yet to be solved, and will probably require a protocol which looks significantly different from BitTorrent.
2) Forward successful download stats to originators... by gsfprez
Many freeware/shareware folks like to keep download stats for marketing purposes, so P2P software and mirrors really irk them....
In order to foster more love from freeware/shareware distributors, could BitTorrent be made to inform the end user (me) that BitTorrent was going to send a "notice of download" (not including any personal information, such as an IP, etc) upon sucessful download (that I could preview before sending of course)?
If *I* was Warner Bros, and eveyone offered to distribute and pay for all the bandwidth for the next version of the Animatrix, while I still got to see download statistics, i'm not sure I'd even would need to provide a direct link to the 150 meg QuickTime files.
With this kind of feedback mechanism, the software/media providers get all the love - download stats, far far far less bandwidth used - and we get all the goodness - their free movies, software, freeware, data, etc. Its the ultimate mirror.
Or am i missing something?
Bram:
I'm happy to report that you are, in fact, missing something. Clients report very detailed statistics to the BitTorrent tracker, including the number of complete downloads and the total amount each peer uploaded and downloaded. If you host a file using your own tracker, all of this data is readily accessible, the same as if you hosted it via http.
By the way, many people find out about tracker statistics reporting and falsely think that hacking their client to exaggerate their upload rate will increase download speeds. Clients actually decide who to upload to based strictly on the transfer rates they experience directly; Tracker statistics are never even sent to them.
3) Comparison to other P2P... by jfmiller
As far as I can tell the genius of BitTorrent is allowing peers who themselves do not yet have a complete file to share the parts they do. With all dew respect to the effort taken, the rest is just functional glue that allows the system to work as it should.
The eDonkey protocol used the same basic premise. How is BitTorrent different to it and other P2P protocols and why did you make that choice?
Bram:
That 'functional glue' is extraordinarily difficult to get to work well. Ever-changing network conditions and very high rates of peers disconnecting produce a very thorny logistical problem. Most existing swarming implementations don't even manage to fully utilize all the upload capacity available to them.
That said, there are other decent swarming implementations. For example, the one in eDonkey is quite serviceable, and Furthurnet's works okay as well. BitTorrent handles the little details of file transfer better than all of the others, but if that were the only difference its advantage would be relatively minor and subtle.
What sets BitTorrent apart is its very robust technique for rewarding specifically the peers which upload the most, known as leech resistance. On the highest level, this prevents a long-term meltdown of the system from being caused by people running leeching clients. It also causes upload and download rates to be somewhat correlated, so peers on good pipes get decent download rates, which increases general good feeling about how the system behaves. Overnet, the follow-on to eDonkey, may start using BitTorrent's peer protocol in the future specifically for the leech resistance properties.
By the way, people sometimes run clients hacked to not upload at all and still experience good download rates. Usually this is because they're downloading a file which has been available for a while and there are many clients which have finished downloading but been left running, so there's plenty of excess bandwidth to go around. Not uploading in a swarm which is still ramping up is generally ruinous for download rates.
4) Improvements... by BJH
Bram,
Do you have any plans for improvements to BitTorrent to improve some of its (few) weaknesses, such as searching for torrent files, bandwidth usage by trackers and inability to download if the tracker goes off the air?
Bram:
I have no plans to add search functionality, since that can be handled at a higher layer, such as google, and finding content via links is considerably more versatile and widespread than keyword searching anyway.
Bandwidth used by the tracker is currently around 1/1000 the total amount of bandwidth used. With some tweaking, I can get that down to around 1/10,000. Going lower than that would require sacrificing the tracker's ability to collect statistics, since those get significant at that scale.
Relying on a single tracker is really no different than relying on a single web site. Any well-colocated machine is plenty reliable enough, and if you really need failover you can do it at the DNS level.
4a) Re: Improvements... by ichimunki5) Impending doom... by damuI would like to refine this question because I have some specific nits that I'd like to pick: why doesn't the client/server open a single port and listen on that instead of opening a new port for each file? Second, why don't the peers maintain and share information about other peers once the download has started-- going through the central tracker provides a central point of failure. Wouldn't decentralizing allow for a .torrent file to have a list of seeds, and then each of the seeds would be able to share information about peers, eliminating the need for a tracker altoghether?
Bram:
Single port has been high on my list of things to do for a while now but keeps getting put off as more immediate concerns pop up. It mostly hasn't been done yet for a highly technical reason. The way BitTorrent currently shuts down is with a hack where the entire event loop is terminated; To support multiple downloads a cleaner technique which only stopped events and sockets related to a particular download which one of them terminates would be necessary. This is reasonably straightforward to implement, but requires a lot of surgery.
By the way, my mail load has made getting actual development done rather difficult as of late. I'm hoping to offset this with contributions from other developers. While there's been plenty of interest in contributing, and a significant amount of contribution to the tracker, to date noone other than me has made any significant changes to the core download functionality.
If anyone really wants to make a significant development contribution to BitTorrent, you should read over the codebase enough to understand it all (the irc channel can be helpful with this) then ask me what's on the to do list. I suggest you do not start implementing your own BitTorrent client. There are already several of those being worked on, and they're all very far from being as mature as the main line client. What's really needed is more development on the main branch.
Are you taking any precautions for your clash with the RIAA/MPAA?
Bram:
I don't expect to run into any legal trouble. BitTorrent can be used for any kind of content, and several web sites have used it for their own files. Also, all the etree usage (live show recordings of bands which permit it) is completely legal. BitTorrent's total bandwith usage would be quite substantial even if the etree distributions were all it was used for.
6) Future Considerations... by pgrote
Do you feel that BitTorrent's core functionality can one day be integrated in the operating system as a file system? The ability to share files among disparate systems in remote locations can be seen as extension of what was started with HTML, et. al.
Bram:
No. BitTorrent's API is one of starting a download and later being notified that the whole download is complete. File system APIs very specifically involve open(), seek(), read() and write(), which are completely different and wholly incompatible with the way BitTorrent works.
The same is true of http by the way. Attempting to make certain protocols act like local file file system access is kludgy at best, both as a literal concept and as a metaphor.
7) Panhandling for internet dollars... by Matey-O
You've got a paypal dontation button to help compensate you for your non-trivial expenditure of time...how well is that working? Is it an adequate revenue stream, or just enough for a pizza or two?
Bram:
So far, more than a pizza, but less than a living. The donations definitely help though.
8) Re: most obvious question... by Noksagt
...what do you think of what people have done with what you have created. I'm sure you might be sick of people asking you how to obtain a torrent for the latest movie, but are you troubled that it is being used for copyright infringement? Pleased? Apathetic?
Do you wish that it was used more for distributing legal ISOs and other files? If so, do you believe you should promote it more for this purpose or promote development of tools to push it in this direction (perhaps automatic creation of torrents on a successful build, etc.).
Bram:
I'm amused mostly. I find humans highly entertaining.
My attempts to promote BitTorrent for any specific purpose basically failed. It's grown almost entirely through guerilla marketing. That said, I'm hoping that in the future BitTorrent starts being used directly by content producers to distribute their own works.
9) Success... by pgrote
BitTorrent has seen a wide array of usage since it debuted. Many have been surprising and it has caught the fire that makes sofwtare a success. How do you personally measure the success of BitTorrent? Has it achieved the goals you first set?
Bram:
I generally measure software success by how many machines it's deployed on. In that sense BitTorrent has done very well, but it will probably become much more widespread as publishers make their content available using it. My current hope is that BitTorrent will one day be installed on almost all end user machines.
10) Commercial Interest... by Noksagt
I think that bittorrent can be of significant commercial interest. It might be used for software updates for instance. Have you pursued this path or have companies approached you? I certainly hope you'd keep a free version available, but a more feature-rich version would surely land you a great deal of money with the right pitch.
Bram:
So far there hasn't been much commercial interest, but I expect that to change now that large deployments have proven the technology so dramatically.
Starting a business is very tempting. BitTorrent has the potential to create such incredible amounts of value that if I manage to make even a tiny fraction of that I could do very well.
-----
one day, just maybe, he'll even make a living from it...
Bram hopes to make a living off code that he wrote that the community seems to really like? Queue the peanut gallery with cries of "sell-out" and "greed" and random smatterings of the words "corporate" and "freedom". I've not used BT extensively but what little I've seen impressed me immensely. Hopefully he can turn it into something that funds its own improvements, and if he's lucky to help pay some bills as well.
"I don't expect to run into any legal trouble. BitTorrent can be used for any kind of content, and several web sites have used it for their own files"
This hasn't seemed to stop them in the past... The RIAA even admitted that at least 15% of Napster use was legal, more than the amount of legal use they admitted in the betamax case...
I think he should start saving up those paypal donations fo the legal fund because in all likelyhood he is going to need it!
Visualize the world of wine
... Are you concerned that M$ might decide to make a P2P system that works similarly to this and start bundling the client with Windows, or even as a part of their Media Player? They would then license the tech to the media companies to use for distributing movies, etc. Such a move could dramatically reduce the growth potential of BitTorrent (see "Netscape" and "RealPlayer").
Even if M$ just gave it away at first in order to take the biggest chunk of the market for later reaping, the impact on other products such as yours and Kazaa and others would be, well, bad.
Flash is the Herpes of the Internet.
your.opinion >
"He already answered this to a large extent, in an essay on Advogato, How to Write Maintainable Code.
I think that the best language is the one that he can maintain, understand, and use. Sure if it was written in pure assembly it would be faster, but it's a bitch to maintain. The clients work reasonably well. I've never had the official client crash on me. It works up to a fairly large scale with decent hardware and bandwidth.
He said images on a website are too small to bother with. Well, how about a tarball of the entire site?
... torrent://slashdotted.site.com
With the home page at the front. It would be terrific if people didn't have to fear being slashdotted. It would be cool if an Apache module could be developed to detect when bandwidth reached over a certain level, made a tarball and only allowed Torrent download of that. Then later, reverted to normal.
Making a file system driver for BitTorrent - not possible too different? I don't buy that. I could be done. Of course, there'd be latency.
Perhaps not handy of interactive use.
Also, how about new bowser protocol tag (like http://)
Since Gnome's VFS already does smb: etc this
would be a nice place to add it.
- Rapid development. For anything (like a P2P app) where the value increases mostly through network effects (how many other people are using the same product), translating ideas into working code is critically important.
- Built-in, conveniently accessed hashes. (aka associative arrays) I program in java regularly, and it's a much bigger pain using a
.get() call for every damn dictionary access than the same code in python. (or perl, for that matter) Any piece of syntax that gets in the way of seeing the forest for all the damn tree leftovers cluttering up your view should be eliminated.
- Cross-platform independence.
As to your objections to python, I find them bizarre. Perhaps on the tracker of a very busy download the CPU speed/memory usage is significant, (witness his statement that several alternative trackers are being worked on) but in any client the entire process is completely network IO bound. As for the problems with Python's garbage collector - BitTorrent (well, the code of it that I've looked at so far) doesn't use recursive structures, so I don't see what the objection is there.Also, "awkward approach towards object creation/destruction"? What exactly does Python do that you wish it didn't? It sounds to me like you're pining for the days before people realized that object destruction costs, and that therefore it's best left to a garbage collector that can operate at idle times.
Most of them, sure. But it just takes one litigious dork who doesn't want his stuff mirrored to ruin it for the rest of us. I agree in general that slashdot should do this, but the details of getting the permission to do so could badly slow the story submission process. At present, I don't think they know how to get out of this predicament, hence, no mirrors.
Insert Witty Remark Here ===>____________________________
not it doesnt. it just needs the small torrent file with the tracker info. person that uploads the .torrent has to have the copy of it first then it spreads.
Currently I download some files using bittorrent (wolfenstein enemy territories) but when all the seeds go away it can cause issues.
WET is my 2nd torrent experience, and the second bad one. I have an ADSL (3M/128k). On the length I have let Torrent downloaded (nearly 3h for 34%), I have get my upload capped at 12k nearly all the time, but my upload has runned at 4k in average with spikes at 10k. With gamespy with 42 min of waiting in the queue, I have downloaded it in less than 2h... I don't remember the exact stats from my first download, but they was as bad as this one... Why everyone say bittorent is the next big thing, but I can't get it to seem work correctly ?
Python can also be fairly well optimized; I have a strong suspicion that the slashdotting of trackers is more a bandwidth issue than a processing capacity issue. It's also quite possible (caveat: I haven't read the code yet) that the internal structure placed a higher priority on readability and maintainability than on processing efficiency - I know I've written server applications using that approach, because I'd rather spend an extra $500 on hardware upgrades if more power is needed than an extra $5000 later on additional maintenance, debugging and testing.
Finally, on the issue of speed: a lot of what a tracker does involves managing lists, hash tables, etc. - the type of thing that's built into Python and highly-optimized. It's like someone's discovery when VB5 or VB6 came out that a VB program was faster at some tasks than a corresponding C++ program, because all the processing was being done using very heavily optimized builtins - if you're running a relatively inefficient scripting language but using it to call highly-optimized native code, you may not see the performance degradation that you might expect.
fencepost
just a little off
I don't know how an honest question regarding bittorrent can be considered offtopic...
You aren't the first person to point out Slashdot's obliteration of all things tracker and get modded down for it! But on a more serious note, just get the original seeder to make a new .torrent file with the new tracker information in it (or anyone that has completed downloading the file). Their is stuff to do it on Brahm's site.
The problem with slashdot is that most of its users were bullied and stuffed into lockers as kids!
Except nothing new has been added in nearly a month. There is no way for folks to upload their own torrent to react in a reasonable amount of time to the speed that the slashdot effect takes place.
Mad Software: Rantings on Developing So
Is the SHA1 actually coded in python, or in C libraries used by the python interpreter? I wouldn't be surprised if it's the latter, in which case recoding the rest isn't much help.