Open Content Network (P2P meets Open Source)

Orasis writes "The creators of Swarmcast have announced a new peer-to-peer content delivery network called the Open Content Network. The OCN will allow users to download open source and public domain content from multiple peers and mirrors in parallel. The system is designed to augment the existing mirrors with bandwidth from the p2p network and should eliminate the "Slashdot Effect" for popular open source content."

Comments

OK, but...

[scrm]

The Open Content site just announces a list of intentions. Anyone can put this kind of info up. It looks to me like nothing has been achieved yet, making this not really news.

Openft

[z-man]

What about the openft protocol, they've been working on that for a while gift.sourceforge.net. They originally used the fasttrack protocol (KaZaa), but after kaZaa changed there specs, they decided to create their own protocol.

Uninformed

[BlueboyX]

Maybe you didn't notice that these guys are the makers of Swarmcast. Or maybe you posted before figuring out what that meant.

Swarmcast is a (working!) program for parallel p2p file downloading. In other words, the technology IS implimented. They basically are just making a modified program to work with a somewhat different set of files. No biggie.

Re:Need for Checksumming

[popeyethesailor]

It is part of the specs

1.2 Untrusted Caches

It is currently unsafe to download web objects from an untrusted cache or mirror because they can modify/corrupt the content at will. This becomes particularly problematic when trying to create public cooperative caching systems. This isn't a problem for private CDNs, like Akamai, where all of their servers are under Akamai's control and are assumed to be secure. But for a public CDN, the goal is to allow user-agents to retrieve content from completely untrusted hosts but be assured that they are receiving the content intact. The CAW solves this problem by using content addressing that includes integrity checking information.

Re:Guesses as to how long this is going to last?

[BCoates]

NAT alone is not an effective method of preventing people from using p2p programs. All it does is prevent incoming TCP connections, so as long as someone in the network (well, some reasonable minority of peers) can get incoming connections to bootstrap people into the network, everyone can still comminicate despite the inability to get new incoming connections.

Good NAT bypassing is annoying to program (in the extreme case, it requires implementing something like TCP over UDP) but it's not a huge techncal hurdle, the main reason it's not commonly done is because too few people have hostile NATs for it to be worth the effort.

--
Benjamin Coates

Re:The perfect task for Freenet

[delta407]

Throughput - Browsers will be able to download content from multiple sources in parallel
Bandwidth Savings - Browsers will automatically discover and select the closest mirror for a piece of content.
Fault Tolerance - Even if a site goes down in the middle of a download, browsers will automatically locate another mirror and continue downloading.
Scalability - Any number of machines may be added to the network, creating a CDN ad hoc, with very little administration.
Security - Browsers will be able to safely download content from untrusted mirrors without risk of corruption or viruses.

Right on target. Freenet accomplishes these goals, and actually works right now. Freenet is essentially an anonymous, distributed caching system into which anyone can insert data and retrieve it later. It supports both locating information by content hashes or by a human-readable redirect, as well as lots of really cool features like anonymous websites ("freesites"). So... what are you waiting for? Install Freenet today!

</plug>

Re:Eliminate the slashdot effect? bah!

[Salamander]

If the goal here is really to eliminate the "Slashdot Effect" a much more effective solution would be to set up a network of load-balanced caching proxies on geographically distributed fat pipes.

Who pays for all that equipment and bandwidth? The idea here is not to solve problems by throwing resources at a problem, but rather to solve them by using existing resources as effectively as possible. The technology involved can be applied to any resource base. The technology-intensive approach using almost-zero-cost resources might well make significant headway against the Slashdot Effect, even if you still think your capital-intensive approach based on older technology is even better.

Another factor you seem to've overlooked is that software like CAW or BitTorrent are distributed for reasons beyond scalability. For example, consider the inherent attack-resistance characteristics of a highly distributed P2P network, vs. your centrally-administered servers. There are other goals as well, such as avoiding legal culpability or financial dependence on corporate benefactors to provide the systems and bandwidth. Whether you agree or disagree with those goals, the fact remains that many people believe in them. Networks like you describe are old hat, dozens have been deployed already, and yet a lot of people still want something different. You've proposed a solution to a different problem than the one Onion Networks et al seek to solve. There's a term for that; we call it missing the point.

BitTorrent

[bramcohen]

BitTorrent enables downloaders to send pieces to each other when they have an incomplete file, making almost unlimited scaling possible. Simple multi-source downloading can be good for performance, but still is limited by the server's upload capacity.

We've had several large deployments of files which are a couple hundred megabytes and up, getting sustained downloads of a couple hundred downloaders at once, serving off a dsl line, and it's worked well.

By the way, BitTorrent, Swarmcast, and OCN all check secure hashes under the hood, so data integrity isn't an issue.