Slashdot Mirror
P2P Content Delivery for Open Source
Orasis writes "The Open Content Network is a collaborative effort to help deliver open source, public domain, and Creative Commons-licensed content using peer-to-peer technology. The network is essentially a huge 'virtual web server' that links together thousands of computers for the purpose of helping out over-burdened/slashdotted web sites. Any existing mirror or web site can easily join the OCN by tweaking the HTML on their site."
It's open source peer-to-peer and handles exactly the problem of distributed serving.
...have been doing this for awhile now. I've seen download sites giving you the option of grabbing game demos from their site or through some small P2P client that they offer, which snags parts of the file from other users and combines them all on your end.
This technology's out there, but it's nice to see it gaining some fairly widespread adoption.
levine
This is a great idea that's been a long time coming. It sounds to me like it takes the ideas first put forth in FreeNet (which spawned later P2P networks like Napster and Kazza) and finally makes them accessible to everyday content producers and consumers.
I'm wondering if maybe this is the future of blogs like Slashdot, with design, features, and content distributed the same way moderation and commenting are today. Creative Commons licensing would be a further boon.
This sort of next generation P2P network might be the weapon we need against the forces of evil, if only we are brave enough to use it.
Boromir, son of Faramir, King of Gondor and Minas Tirith
I see a slight problem, depending on how CAW is implemented.
Scenario #1:
Assuming the Originator Apache responds with HTTP headers such as those in CAW to advertise site-wide mirrors like this:
X-URI-RES: http://urnresolver.com/uri-res/N2L?urn:sha1:; N2L
When the originator Apache site updates any documents, the URN resolver (or mirror) will silently fail without realizing which document has been updated. It would need to rescan the entire website, even when only one document has changed.
Scenario #2:
The opposite problem occurs with the Originator Apache responding with HTTP headers such as this:
X-URI-RES: http://untrustedmirror.com/pub/file.zip; N2R
The mirror will respond successfully, but will give an out-of-date version of the file without the client or the mirror realizing it. The mirror would then have to manually scan the website on a regular basis (even when nothing has changed) to prevent anything getting too out of date.
Scenario #3 (Solution):
However, if the Originator Apache responds with HTTP headers such as this:
X-URI-RES: http://untrustedmirror.com/pub/file-mirrors.list; N2Ls; urn:sha1
When the URN resolver or Mirror sees the SHA-1 hash mismatch, it knows which document needs to be updated, and can respond by doing so for just that document.
I realize that CAW is mainly designed with static files in mind (images, PDFs, ISOs) where updates occur rarely (or never). And no, I don't see Apache calculating the SHA-1 for dynamic pages like Slashdot anytime soon. However, updates do occur to images, PDFs, ISOs, etc. on occasion. I do think CAW(#3) could be used (and useful) for large, heavily subscribed RSS feeds without too much trouble. Maybe elsewhere in dynamic content.
"There's so much left to know/ and I'm on the road to find out." -Cat Stevens
What good are these CPU hogging, network lagging programs if they aren't delivering pirated software and p0rn? I won't stand for this abomination!
Well, at least until the next time I need to download the newest slackware...
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
I don't see this helping the slashdot effect. How many people are actually going to download the browser plugins required to make all this work?
I mean, I might get the plugins if I'm dealing a lot with sites that use this technology, but how many people will be dealing with a lot of these?
And those sites are using this, are probably the ones that are use to high volumes of traffic, so they prepare for it. The average site that can't handle a slashdot, can't handle it because they generally don't need to.
indviduals will be able to help distribute free content by donating their spare bandwidth and disk space to the network.
Sarcasm aside, while I can see where they're going with this I can't see it ever seriously taking off. Most of the world are still on 56k (or less) and I know I regularly hunt for things to delete so I can squeeze something else on my hard drive.
You'd run a client on your machine that would act as a local DNS server. Then you'd point your machine to this DNS server. So when you goto a site (say off of slashdot) the DNS server would interact with the P2P network and give the IP of the less loaded machine in the P2P network. Yeah, you'd have to run a deamon on your machine, but oh well...
Yes Francis, the world has gone crazy.
I always wondered if it were possible to share people's browser cache contents via P2P technology (with exceptions for the secure documents, of course).
I guess the big problem is still with the indexing.
How far away is this from a
p2p://www.cnn.com/
style link for Explorer/Mozilla/Opera/Konqurer?
Turn everyone's browser cache into p2p.
CNN's probably a bad example, as the content would have to be updated more frequently... And you'd need some way of having a "revision model", so that sites could be updated. I guess it would be up to the clients to ditch old versions of pages.
Might also need some sort of (eep!) central authority to verify pages were who they claimed to be (so I couldn't take over CNN, for example). Maybe just signed keys for each content provider would be good enough?
When I wanted to download a Linux distro, P2P was the first place I looked. I didn't want to cost the providers a gig of traffic when they're not making any money on it. Pity I didn't really find what I was looking for. This was a while back, though.
I'm the type of guy who doesn't like sharing my bandwidth, but I'd be willing to make an exception for Open Source stuff just on the grounds the it helps alleviate the costs of hosting free stuff.
"Derp de derp."
I looked over the website and the site for the current client, and found only faint, inspecific references to what loading such a client does to your machine and internet connection.
This is terrible.
We complain when Gator is loaded as an 'add-on' to our system, yet we don't mind if we are not allowed to download some content without loading some P2P app which then uses our disk space and internet connection to serve others?
They need to put up a specific message that says, in effect, "This download client will significantly speed up the process of obtaining this file. Once downloaded your computer will allow other people to download this same file, or portions of it, from your computer so they can gain the same speed benefit you will get. There is no security risk, and you can stop the client from letting others download this file by moving or deleting the file, or ending the client by doing x, y and z. If you wish to simply download the file normally without installing this client, click here - otherwise click 'OK'"
Yes, we all understand what P2P means - we are donating part of our computer and network to the P2P network for as long as we are connected to the internet. But this is not common terminology - ask a non-computer expert who has spent hours downloading music from their favorite P2P app what the P2P app does, and all they know is that they can get "free" music with this cool program. They often have no idea that others are downloading music from their computer, etc.
This may slow down adoption, but the reality is that the backlash that may come out against it is not worth the extra adoption it may gain without full and well-explained disclosure - as well as a method to download the file normally.
-Adam