Slashdot Mirror
Swarmcast GPLed
miguel writes "OpenCola has just released SwarmCast which is a very interesting mechanism for distributing software. At Ximian we are looking into integrating this into Red Carpet to accelerate software downloads by using their sharing software. The demo of their product is pretty amazing." Very clever: essentially it creates a peered network so larger files can be shuttled around faster. Each client can serve a small piece of data to other clients so that a massive centralized data center isn't necessary. Now the cola on the other hand...
Swarmcast is neither a fragile chain structure, nor a hierarchy, it is a many-to-many 'swarm' structure where peers send and recieve data from many peers in parallel. The use of Forward Error Correction allows us to have a potentially huge number of unique packets in the mesh where only a small subset of those packets are needed to recreate the original content. This allows the peers to swap data back and forth in a fairly random fashion to provide a high level of resiliance against changing network conditions, very high throughput, and rapid scalability.
-Justin Chapweske, Lead Swarmcast Developer
Everything you need to both serve and download content is released under the GPL. Besides, its peer-to-peer so there really isn't that much of a "server" concept. The gateway is mostly for content management and permissions, the kind of stuff that companies pay money for so that I can keep my job and write more open source code. -Justin Chapweske, Lead Swarmcast Developer
Man, you don't usually see download rates like that from anything except akamai. Problem was, the data got corrupted.
And no, it's not like a bucket brigade. It's more like building a house with more than one bricklayer.
Re: slow hosts-- if I understand correctly, all swarmcast hosts maximize their available bandwidth, automatically balancing the download.
Each node downloads packets one at a time. If a host is slow, it won't grab as many packets, and someone else will.
If machines A and B are downloading, and A is 5x as fast as B, then in the same time, A will download 5 packets and B will download 1. If there are 12 packets in the file, A will download 10, and B will download 2.
First of all, what type of security is going to be implemented to prevent hacks. It seems that it would be pretty easy to shore up a single server, or even several in a single datacenter, but it would be a daunting task to protect thousands of machines spread thoughout the world against hackers.
So long as the swarmcast client software doesn't have any holes, how is this an issue? If it does have holes, yes, this is a big deal, but if you've got a machine on the net with other security holes, that's an entirely different problem.
Secondly, what type of redundancy is going to be built into this system. Again, if a file is going to be served from a single centralised machine, it should ideally be fairly reliable, with multiple connections, and RAID to ensure continuous uptime. However, if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.
This is a pretty trivial problem to solve. There's still got to be a central server somewhere that tells you where to look for the peer bits. I don't know the details of their setup, but it certainly seems trivial to implement MD5-checking per chunk, so you can tell if there are bad chunks in your download. You just have to trust the main server's md5 signatures, and never end up trusting the peer servers.
This applies to the first point, too. If a serving machine is hacked, it's not a problem beyond that box because the real security comes from the MD5 key, or equivalent. It may be possible for a hacked server to serve bad code, but, given reasonable client design, it should be impossible for this bad data to actually be used by another machine. (Or, at least, should be impossible without big flashing lights and sirens screaming "You fool, don't do that!" :)
Secondly, what type of redundancy is going to be built into this system. ...if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.
One of the main benefits of distributed anything is that there's an incredible amount of redundancy because every node is willing to contribute a little towards achieving your goal.
Are you trolling?
First of all, what type of security is going to be implemented to prevent hacks.
Against what? To prevent against file modification, checksums can be distributed from a single point since they're much smaller. All other security problems will be the same as any other P2P system.
--
Why not distribute the pieces on Freenet? Using SSKs you can be sure that each piece was inserted by the original poster. Maybe this would make a cool client for Freenet, though...
Can your IM do this?
Still a cool idea, but if OpenCola wants everybody to put eggs in their basket, maybe somebody should release a GPLed server too...
--
314-15-9265
Well, my first problem is that the FTP server for downloading the JWS file seems to be /.'ed, so I can't get that and can't install SwarmCast, which would have solved the problem in the first place. Hmph...ironic, in a Catch-22 kinda way, isn't it? ;)
nlh
Ferrari and other exotic car rentals in New York
Sure, their client may be under the GPL but some of the terms on their license agreement are quite unpleasant:
(a) Reverse assemble, reverse compile, or reverse engineer the Gateway (or any component or portion thereof), or otherwise attempt to discover any underlying Proprietary Information (defined below) of the Gateway;
Isn't reverse engineering explicitly granted by law?
(b) Sublicense, rent, sell, lease or otherwise transfer the Gateway (or any portion thereof) to any third party;
(c) Remove or alter any marks or designations indicating the ownership of copyrights, trademarks or other intellectual property rights of any party contained in the Gateway;
In fact I'd go so far as to say these conditions wouldn't feel out of place in a Microsoft EULA.
The EFF is seeking help in this area of finding legitimate uses of peer to peer technologies...
Maybe this might be a very good argument in court... content distribution at high speeds.
Ever need an online dictionary?
This saves the content providor bandwidth at my expense. Bandwidth is, of course, not free and somebody has to pay for it. Ximian take note, if you don't want to pay for the bandwidth involved in being in the software distribution business, get out of that business. For Joe Average that installs Red Carpet and doesn't understand that it now includes this new feature, is going to be mighty pissed when he gets his bandwidth bill only to discover that he's been serving 10 gigabytes to strangers due to this feature.
A journey of a thousand miles starts with a brutal anal raping at airport security
First of all, what type of security is going to be implemented to prevent hacks. It seems that it would be pretty easy to shore up a single server, or even several in a single datacenter, but it would be a daunting task to protect thousands of machines spread thoughout the world against hackers.
Secondly, what type of redundancy is going to be built into this system. Again, if a file is going to be served from a single centralised machine, it should ideally be fairly reliable, with multiple connections, and RAID to ensure continuous uptime. However, if you're serving tiny pieces of a file from thousands of boxen, it seems to me that if even one of those fragments doesn't make it all the way downstream, the whole file would be worthless. Has Swarmcast done anything to prevent this from happening.
-atrowe: Card-carrying Mensa member. I have no toleranse for stupidity.
You've successfully quoted the Sun Java Web Start license. Now please click the word 'license' in Swarmcast's about box and you will see that there are no such reservations or claims.
You are the weakest link. Goodbye!
Thing is, I don't see why anyone would use this. In reality, the transfer rates aren't as good as a single fast dedicated server (I can easily get 75-80KB/sec on this line), so there's really no gain on the client end from using this. On the server end, yeah you're using a lot less bandwidth. This might be useful for open source projects or other products that are downloaded by knowledgable people, but you're average computer luser isn't going to want to download 6MB of Java Runtime/Swarmcast Client just so they can save your company money by getting slower-than-normal downloads. (And I shudder to think what a distributed p2p network comprised mostly of 56k modems would be like, at least at the moment it seems most of the users online are broadband.)
Anyways, pessimism aside, here's what I'd like to see:
OK, I'm done for the moment. If they play this right I can see it helping out smaller outfits with knowledgable users, maybe eventually even going mainstream if they can convince people it's worth their while to install. It's definitely sparse on information in it's current form, but hey, it's a beta. It's a good idea, and I wish them luck.
-Jade E.