No way is Google going to spend the capital to do their own satellite system or the licensing fees to use someone elses. They'll be doing it over broadband to a hard drive within the Set Top Box.
If they want this thing to be cost effective for HD, they should use Swarmstreaming.
Sure you need to be able to take a loss on it for a while, but the original poster is correct that you need to have customers driving your requirements from day one.
Zombie Pub Crawl Video
on
Zombie Lurch
·
· Score: 1
The Minneapolis Zombie Pub Crawl last weekend was one of the most fun times I've had in years. Someone made a little documentary type video of it:
Hotlinking is theft? You've got to be kidding me. Sure he has the right to do what he wants to with his files, but that doesn't change the fact that he's dealing with the situation in a completely immature fashion.
While the tone of this post suggests that Tor is in some way inferior to Freenet, this couldn't be further from the truth. Tor works TODAY and allows you to proxy any SOCKS-compatible application (damn-near every networked app) through the anonymizing network, this includes SSH, e-mail, web browsing, etc.
Freenet on the other hand is built for transporting bulk data in an anonymous fashion and is thus probably better suited to P2P file sharing - it will be impossible to ever tunnel SSH over Freenet.
For swarmstreaming, we use the Tree Hash EXchange format (THEX) to provide cryptographic integrity verification down to a single 1KB resolution so we can automatically repair the corruption.
The concept of "swarming AI" is completely different than "swarming downloads". I used to always fully qualify it as "swarming downloads", but people have just taken to calling it swarming nowadays.
The concept of swarming in AI has indeed been around for years and I think its some great stuff. However, ant optimization algorithms have nothing to do with swarming downloads or P2P content delivery.
This is exactly what Brandon Wiley's Alluvium is doing, except that it uses Oggs instead of MP3. Its basically an implementation of the whole "Judo Radio" concept where you download and cache the files ahead of time and just receive a playlist that tells you the order the files should be streamed.
Swarming in the context of AI has of course existed for a long long time and I of course did not invent that. What I invented was swarming downloads with Swarmcast, which is the technique for P2P/grid file transfer.
Yup, this guy hit the nail on the head. We've actually been doing swarming+progressive downloads since 2001. The big change with swarmstreaming is that this stuff scales to huge numbers of files in a single cache and supports out-of-order random access, which turns out to be much harder than progressive playback is. So if you have an application that only needs to read small ranges of bytes in a huge file, you can now use swarmstreaming to do the trick.
Heh, thanks. Actually, I think this whole situation highlights just how much current client/server web architecture sucks. It just puts a fire up under our asses to get this technology adopted on a wide scale that much faster:)
Technically, swarmstreaming is more closely related to progressive playback than RTSP style streaming, since it relies on a full file being available at an origin web server.
One problem that I see with your proposal is that servers would need to keep track of all connections in times when they're overloaded anyway to properly enforce the mechanism.
Not true, we've already implemented something like this and it works just great. You simply allocate a fixed amount of memory to keeping track of aggressive clients and use an LRU structure to forget about the less aggressive ones.
The main problem here is that RSS lacks any sort of distributed flow control, much as the Internet did back in the early days with tons of UDP packets flying around everywhere and periodically bringing networks to their knees.
One completely backwards-compatible fashion to add flow-control to RSS would be to use the HTTP 503 response when server load is getting too high for your RSS files. The server simply sends an HTTP 503 response with a Retry-After header indicating how long the requesting client should wait before retrying.
Clients that ignore the retry interval or are overly aggressive could be punished by further 503 responses thus basically denying those aggressive clients access to the RSS feeds. Users of overly aggressive clients would soon find that they actually provide less fresh results and would place pressure on implementors to fix their implementations.
Slashdot Mirror
No way is Google going to spend the capital to do their own satellite system or the licensing fees to use someone elses. They'll be doing it over broadband to a hard drive within the Set Top Box.
If they want this thing to be cost effective for HD, they should use Swarmstreaming.
Thats basically it for me.
Sure you need to be able to take a loss on it for a while, but the original poster is correct that you need to have customers driving your requirements from day one.
The Minneapolis Zombie Pub Crawl last weekend was one of the most fun times I've had in years. Someone made a little documentary type video of it:
video
Hotlinking is theft? You've got to be kidding me. Sure he has the right to do what he wants to with his files, but that doesn't change the fact that he's dealing with the situation in a completely immature fashion.
Now we're starting to see his true colors it seems....
Use SRP, B-SPEKE, or any of the other hundreds of variations of secure password authentication that have been invented.
While the tone of this post suggests that Tor is in some way inferior to Freenet, this couldn't be further from the truth. Tor works TODAY and allows you to proxy any SOCKS-compatible application (damn-near every networked app) through the anonymizing network, this includes SSH, e-mail, web browsing, etc.
Freenet on the other hand is built for transporting bulk data in an anonymous fashion and is thus probably better suited to P2P file sharing - it will be impossible to ever tunnel SSH over Freenet.
For swarmstreaming, we use the Tree Hash EXchange format (THEX) to provide cryptographic integrity verification down to a single 1KB resolution so we can automatically repair the corruption.
The concept of "swarming AI" is completely different than "swarming downloads". I used to always fully qualify it as "swarming downloads", but people have just taken to calling it swarming nowadays.
The concept of swarming in AI has indeed been around for years and I think its some great stuff. However, ant optimization algorithms have nothing to do with swarming downloads or P2P content delivery.
This is exactly what Brandon Wiley's Alluvium is doing, except that it uses Oggs instead of MP3. Its basically an implementation of the whole "Judo Radio" concept where you download and cache the files ahead of time and just receive a playlist that tells you the order the files should be streamed.
You're in luck: Swarmcast is a transparent caching web proxy server.
Swarming in the context of AI has of course existed for a long long time and I of course did not invent that. What I invented was swarming downloads with Swarmcast, which is the technique for P2P/grid file transfer.
Its not just for media. Imagine a 10GB CAD file that you want to grab a 10MB object out of. You could grab just the bytes you need and skip the rest.
Yup, this guy hit the nail on the head. We've actually been doing swarming+progressive downloads since 2001. The big change with swarmstreaming is that this stuff scales to huge numbers of files in a single cache and supports out-of-order random access, which turns out to be much harder than progressive playback is. So if you have an application that only needs to read small ranges of bytes in a huge file, you can now use swarmstreaming to do the trick.
Heh, thanks. Actually, I think this whole situation highlights just how much current client/server web architecture sucks. It just puts a fire up under our asses to get this technology adopted on a wide scale that much faster :)
Technically, swarmstreaming is more closely related to progressive playback than RTSP style streaming, since it relies on a full file being available at an origin web server.
Related to this, check out the Visual P2P/Swarming Simulation.
It allows you to visualize firewalled transfers.
This is really creepy to hear after playing Doom 3 all night. The pioneer spacecraft is going to comeback in 100 years with a doorway to hell!
One problem that I see with your proposal is that servers would need to keep track of all connections in times when they're overloaded anyway to properly enforce the mechanism.
Not true, we've already implemented something like this and it works just great. You simply allocate a fixed amount of memory to keeping track of aggressive clients and use an LRU structure to forget about the less aggressive ones.
The main problem here is that RSS lacks any sort of distributed flow control, much as the Internet did back in the early days with tons of UDP packets flying around everywhere and periodically bringing networks to their knees.
One completely backwards-compatible fashion to add flow-control to RSS would be to use the HTTP 503 response when server load is getting too high for your RSS files. The server simply sends an HTTP 503 response with a Retry-After header indicating how long the requesting client should wait before retrying.
Clients that ignore the retry interval or are overly aggressive could be punished by further 503 responses thus basically denying those aggressive clients access to the RSS feeds. Users of overly aggressive clients would soon find that they actually provide less fresh results and would place pressure on implementors to fix their implementations.
With all of the greed in high-tech, its great to see some people actually making lives a bit better.
The questions are incredibly vague and guide the survey-taker enough that absolutely no assumptions should be gleaned from the results of this survey.
He is one of my employees at Onion Networks and is an insanely talented developer. His web site is at http://ry4an.org/unblog/
Hopefully he'll get on this thread and respond with his experiences.
Congrats Gojomo!
This project was written by the brains behind bitzi and some really cool P2P stuff.
He's one of those guys thats going to be working on important stuff for years to come.