Slashdot Mirror
Evolving Swarms with Swarmstreaming
Orasis writes "Applications like Bittorrent have broadly validated swarming technology in the real-world. Now, the inventor of swarming has released a new technology called swarmstreaming that allows smooth progressive playback of content, skipping ahead, and random access without downloading the entire file. It's an HTTP proxy, so browsers, podcasting, and RSS apps should be able to use it transparently. "
smooth progressive playback of content, skipping ahead, and random access without downloading the entire file
Quicktime has had all that for several years. Apple called it "Instant On". I think both Real and Microsoft already use something similar.
Here's a link to Onion Networks' Swarmstream product. It appears that to use the product, you need to purchase a Swamstream SDK, though.
Swarmstreaming: Swarming Downloads Evolved .swarm files. This is probably our most exciting advancement since the original invention of swarming.
r mstreaming
I'm proud to finally unveil swarmstreaming our third generation of swarming algorithms that are designed for the fastest downloads of web content and multimedia without any special server software or silly
The technology improves swarming by ensuring that the bytes that the user wants next are scheduled to be received next. So if they're playing back a video file, the bytes from the front of the file will be received first. If the user (or application) skips forward to the middle of the file, the bytes at the middle of the file will be prioritized. Thus, unlike first generation swarming systems like Swarmcast or Bittorrent, you don't have to wait for the entire file to download to do something useful with it!.
Under the covers it is almost unimaginably more complicated than this because it also provides Self-Healing Downloads, implements a full-blown, scalable, Web Proxy Cache, and actively works to ensure that the video playback never studders or buffers by constantly monitoring and adapting to changing network conditions. For a raw feature dump, check out the SwarmStream SDK Feature Matrix
Nowadays, because of its immense popularity, most people have only heard of swarming because of Bittorrent. I have no animosity towards Bittorrent because it has done more than any application to prove the value of swarming to the general public. But if people are impressed by Bittorrent, they're going to be absolutely blow away by swarmstreaming and how far we've taken swarming since its humble beginnings five years ago.
The best source of information right now on swarmstreaming is Onion Networks SwarmStream SDK, so check it out and let me know what you think.
He links to http://onionnetworks.com/technology/swarming/#swa
December 13, 2004
.swarm files. This is probably our most exciting advancement since the original invention of swarming.
Swarmstreaming: Swarming Downloads Evolved
I'm proud to finally unveil swarmstreaming our third generation of swarming algorithms that are designed for the fastest downloads of web content and multimedia without any special server software or silly
The technology improves swarming by ensuring that the bytes that the user wants next are scheduled to be received next. So if they're playing back a video file, the bytes from the front of the file will be received first. If the user (or application) skips forward to the middle of the file, the bytes at the middle of the file will be prioritized. Thus, unlike first generation swarming systems like Swarmcast or Bittorrent, you don't have to wait for the entire file to download to do something useful with it!.
Under the covers it is almost unimaginably more complicated than this because it also provides Self-Healing Downloads, implements a full-blown, scalable, Web Proxy Cache, and actively works to ensure that the video playback never studders or buffers by constantly monitoring and adapting to changing network conditions. For a raw feature dump, check out the SwarmStream SDK Feature Matrix
Nowadays, because of its immense popularity, most people have only heard of swarming because of Bittorrent. I have no animosity towards Bittorrent because it has done more than any application to prove the value of swarming to the general public. But if people are impressed by Bittorrent, they're going to be absolutely blow away by swarmstreaming and how far we've taken swarming since its humble beginnings five years ago.
The best source of information right now on swarmstreaming is Onion Networks SwarmStream SDK, so check it out and let me know what you think.
SwarmStream Development Suite Features
* Object code for the entire suite of SwarmStream(TM) APIs, including WebRAID(TM), DirectCache(TM), Throttling, and THEX.
* Visualization tools to perform live inspections and demonstrations of what SwarmStream is doing during your application run time.
* One full license for WAN Transport(TM) Server (normally $2950), an HTTP server specifically designed provide advanced SwarmStream features such as self-healing downloads and automatic mirror discovery.
* One full day of developer training
* 20 hours of ongoing support
* One year of free upgrades for all of the above software.
* Unlimited right to use and implement SwarmStream technology for testing, prototyping, demonstrations, or creation of reference designs or applications. Production deployment requires an additional Deployment License.
* One-time fee: $25,000
Anyone care that Orasis (the story author) = Justin Chapweske?
http://www.advogato.org/person/orasis/
-c
Here is some info on the new technology from the guy's company's website: http://onionnetworks.com/products/swarmstream/
On a sidenote, I seriously doubt that he is the very first one to have thought of swarming. Swarming has been around since before 1999 (when he claims he invented it). He *may* be the first one to have applied it to p2p/networking however.
There is a link to a presentation about this stuff on javalobby:
http://www.javalobby.com/eps/swarmstream/
The application itself is free for everyone to use, even as a proxy. The license above is if you want to develop an application that includes SwarmStream.
Yup, this guy hit the nail on the head. We've actually been doing swarming+progressive downloads since 2001. The big change with swarmstreaming is that this stuff scales to huge numbers of files in a single cache and supports out-of-order random access, which turns out to be much harder than progressive playback is. So if you have an application that only needs to read small ranges of bytes in a huge file, you can now use swarmstreaming to do the trick.
No, you want to watch from the beginning. However, BitTorrent is not designed for this - it specifically targets the worst represented portions of the file to help make the swarm as diverse as possible, so that as seeds disappear major bottlenecks do not arise. There are implementations like Azureus that can favor the first chunks, but the result is sketchy at best. Specifically delivering a desired chunk has merit anyway - but only for streamable content, and only if it can deliver fast enough for playback.
Also, BT's speed is often bottlenecked by ISPs which cap uploads, therefore penalizing you on the downloading side.
It's right there on the website though - due to the nature of the technology you have to update your playback/reader/displayer app to take advantage of this. BT just works as is. If the BT developers really wanted to they could update the protocol to allow all this, or even more interesting features - like parchive file repair and recovery, which would effectively eliminate the problem of swarms with lost seeds.
---If you can't trust a nerd, who can you trust?
He is not the inventor of "swarming". It is an application of the "ant algorithm" and associated techniques developed by Marco Dorigo.
http://iridia.ulb.ac.be/~mdorigo/ACO/ACO.html
You'd still probably want your nice expensive HDTV for stuff where quality really matters, but as far as delivering high-quality video over the Internet the capacity is definitely there. Well, it's there on the user side: if a million people suddenly started downloading 2.6GB files all at once I can imagine a few of the server's routers running shrieking from the data center while on fire.
mplayer or vlc plugins