Slashdot Mirror
P2P Content Delivery for Open Source
Orasis writes "The Open Content Network is a collaborative effort to help deliver open source, public domain, and Creative Commons-licensed content using peer-to-peer technology. The network is essentially a huge 'virtual web server' that links together thousands of computers for the purpose of helping out over-burdened/slashdotted web sites. Any existing mirror or web site can easily join the OCN by tweaking the HTML on their site."
Can you hide your IP?
That's the only thing that matters!
It's Christmas everyday with BitTorrent.
Although it won't help when another worm creates massive amounts of traffic on the internet...
But at least when the net gets back a usable level, slashdot readers will be able to read about it without slashdotting a site.
What we need is some gateway product to get young kids hooked on programming.
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
It's open source peer-to-peer and handles exactly the problem of distributed serving.
Why not have a system that can deliver web pages(all the content YOUR hard drive can handle) via a P2P system?
I'd love to see unreliable, poorly maintained, pop-up happy free websites like angelfire and geocities go away and use a vastly superior P2p system instead. SOmeone wants to connect to your special web page? Have them connect to you via the P2p client. No need to fuss with slow FTPing into servers to upload/update web content. It's already on your system.
...have been doing this for awhile now. I've seen download sites giving you the option of grabbing game demos from their site or through some small P2P client that they offer, which snags parts of the file from other users and combines them all on your end.
This technology's out there, but it's nice to see it gaining some fairly widespread adoption.
levine
Really. I can go to the distribitors website and click on "download" and save myself all the trouble.
All the best,
--Bob
This is a great idea that's been a long time coming. It sounds to me like it takes the ideas first put forth in FreeNet (which spawned later P2P networks like Napster and Kazza) and finally makes them accessible to everyday content producers and consumers.
I'm wondering if maybe this is the future of blogs like Slashdot, with design, features, and content distributed the same way moderation and commenting are today. Creative Commons licensing would be a further boon.
This sort of next generation P2P network might be the weapon we need against the forces of evil, if only we are brave enough to use it.
Boromir, son of Faramir, King of Gondor and Minas Tirith
I see a slight problem, depending on how CAW is implemented.
Scenario #1:
Assuming the Originator Apache responds with HTTP headers such as those in CAW to advertise site-wide mirrors like this:
X-URI-RES: http://urnresolver.com/uri-res/N2L?urn:sha1:; N2L
When the originator Apache site updates any documents, the URN resolver (or mirror) will silently fail without realizing which document has been updated. It would need to rescan the entire website, even when only one document has changed.
Scenario #2:
The opposite problem occurs with the Originator Apache responding with HTTP headers such as this:
X-URI-RES: http://untrustedmirror.com/pub/file.zip; N2R
The mirror will respond successfully, but will give an out-of-date version of the file without the client or the mirror realizing it. The mirror would then have to manually scan the website on a regular basis (even when nothing has changed) to prevent anything getting too out of date.
Scenario #3 (Solution):
However, if the Originator Apache responds with HTTP headers such as this:
X-URI-RES: http://untrustedmirror.com/pub/file-mirrors.list; N2Ls; urn:sha1
When the URN resolver or Mirror sees the SHA-1 hash mismatch, it knows which document needs to be updated, and can respond by doing so for just that document.
I realize that CAW is mainly designed with static files in mind (images, PDFs, ISOs) where updates occur rarely (or never). And no, I don't see Apache calculating the SHA-1 for dynamic pages like Slashdot anytime soon. However, updates do occur to images, PDFs, ISOs, etc. on occasion. I do think CAW(#3) could be used (and useful) for large, heavily subscribed RSS feeds without too much trouble. Maybe elsewhere in dynamic content.
"There's so much left to know/ and I'm on the road to find out." -Cat Stevens
What good are these CPU hogging, network lagging programs if they aren't delivering pirated software and p0rn? I won't stand for this abomination!
Well, at least until the next time I need to download the newest slackware...
"Learning is not compulsory... neither is survival."
--Dr.W.Edwards Deming
I don't see this helping the slashdot effect. How many people are actually going to download the browser plugins required to make all this work?
I mean, I might get the plugins if I'm dealing a lot with sites that use this technology, but how many people will be dealing with a lot of these?
And those sites are using this, are probably the ones that are use to high volumes of traffic, so they prepare for it. The average site that can't handle a slashdot, can't handle it because they generally don't need to.
How would something like this work? Presumably slashdot would have to link to a single site, which then farmed out the requests to participants? If this is the case, there is still a single point of faliure, right? And presumably browsers need to know that they are being redirected so any subsequent requests. Thus, how is this more powerful than an (albeit intelligent) javascipt forwarder...? If it's just a simple load baalncing system then I don't see what's so groundbreaking.
Vacancy for signature. Apply within.
If that network don't have any kind of moderation (and as a p2p network, not sure how exactly that can be done right), what will stop people to post or ensure that is not downloading a trojanized versions of programs? Or a new way to distribute viruses, like the ones that are already for kazaa and similars.
I for one can not wait until the golden age of computing, an age in which my computer will be inexplicity linked to that of every script kiddie from Taiwan to San Jose. This will be a stunning monument to the ingenuity of man. There will be a great many personal homepages replaced with vivid images of genitalia and beer bottles. What a wonderous age it will be.
indviduals will be able to help distribute free content by donating their spare bandwidth and disk space to the network.
Sarcasm aside, while I can see where they're going with this I can't see it ever seriously taking off. Most of the world are still on 56k (or less) and I know I regularly hunt for things to delete so I can squeeze something else on my hard drive.
To recap:
- Reliable
- Anonymous
- Totally decentralized
- More popular files are more widely distributed thus avoiding any
/. effect
Install a recent snapshot of Freenet, then visit this freesite to check it out.You'd run a client on your machine that would act as a local DNS server. Then you'd point your machine to this DNS server. So when you goto a site (say off of slashdot) the DNS server would interact with the P2P network and give the IP of the less loaded machine in the P2P network. Yeah, you'd have to run a deamon on your machine, but oh well...
Yes Francis, the world has gone crazy.
I always wondered if it were possible to share people's browser cache contents via P2P technology (with exceptions for the secure documents, of course).
I guess the big problem is still with the indexing.
...Google's cache!
Any existing mirror or web site can easily join the OCN by tweaking the HTML on their site
Sounds a lot like it to me - especially the bit about it being like a virtual server. I suppose if it stored images and stuff then it's be a bit better, but will it ever match Google's speed and breadth of content?
I surely see some potential in the idea but what about standards. Allthough it's GPL-ed the standards aren't adopted by any organisation like IETF, OASIS or W3C. If one of these organisations would see the potential for this it would make things a lot easier. You would get rid of plug-ins, extentions etc. and all browsers as well as servers would support it.
In fact I'm pretty astonished none of these organisations has ever picked up P2P.
/(bb|[^b]{2})/
LOL
Damn, where are my mod points. That's too funny
It seems like alot of you just read the front page and seem to think this is some type kazaa clone. I agree the browser pug in is kinda gay.
What this "could" mean is say if your favorite distro has just been updated, we all know how hard it is to download 3 isos while they are in high demand. The thing about OCN and OnionNetworks type software is that the high demand and download rate will help the availibility. Plus everything is authenticated and logged so worms/trojens/fakes really arnt a problem.
As far as OCN goes, it's not for warez, and divx. I think it's intended for geek's free software distrobution. So love it, and try to inovate it.
-makoffee
Codecon - www.codecon.info will be February 22-24 in San Francisco. It's a conference about writing code for applications like peer-to-peer and crypto (and crypto peer-to-peer, etc.), oriented towards authors presenting actual working demos. The program page has abstracts of the talks/demos. Many of these applications overlap some of the same space. One of the organizers is Bram Cohen, author of the BitTorrent P2P file distribution system (and one of the organizers of last year's conference), and the other is Len Sassaman, who does cryptographic remailers.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
How far away is this from a
p2p://www.cnn.com/
style link for Explorer/Mozilla/Opera/Konqurer?
Turn everyone's browser cache into p2p.
CNN's probably a bad example, as the content would have to be updated more frequently... And you'd need some way of having a "revision model", so that sites could be updated. I guess it would be up to the clients to ditch old versions of pages.
Might also need some sort of (eep!) central authority to verify pages were who they claimed to be (so I couldn't take over CNN, for example). Maybe just signed keys for each content provider would be good enough?
It turns my cable modem into a 2400 baud acoustic. The nostalgia wears off quickly.
Indeed. It's a shame no one thought of it sooner.
-- "Government is the great fiction through which everybody endeavors to live at the expense of everybody else."
You can limit the amount of bandwidth Freenet uses if you think it is using too much.
The newly relaunched Happy Puppy uses the Tornado Cache Plug-in to provide some pretty fast downloads. If you're interested in seeing this stuff in action I recommend checking out the site.
That said, the OCN is open to any and all applications out there, so I'd encourage them to join the OCN.
The game site http://happypuppy.com uses software by one of the sponsors of the Open Content Network.
From the website:
"Multimedia files are eligible to be distributed via the OCN if they are either released into the public domain or are available under a Creative Commons license that allows the content to be freely copied."
and
"Software is eligible to be distributed via the OCN if it is released into the public domain or is licensed under an Open Source license. The license must be an OSI Approved License that adheres to the Open Source Definition."
Honestly, those constraints seem to seriously restrict the real usefulness of the network. It means if I want to put up a webpage and publish the contents with OCN, I need to go through all the rigamarole to make sure that everything's copacetic with whatever the "approved" licenses are, instead of just saying "ok, stick this out there." Which I may not want to do because if I've just created some magnificent thing (music file, video file, whatever) I may not necessarily want the license to allow anyone to download it and modify it any way they want and then essentially claim it as their own.
Software is one thing, but online content is something else. Honestly, how many large "media files" have you seen that are licensed under an "Open Content" license?
Sure, it's nice to have something like this that caters explicitly to the OpenSource crowd, but with those constraints, I can't see it as used for very much other than putting new versions of GPL'd software packages online.
The CC License Engine framework really does all that is needed to identify a license that would allow the P2P network to redistribute the material , so why not use it?
Or am I missing something now?
Employee of Inrupt, Project Release Manager and Community Manager for Solid
The primary purpose of the OCN is to promote public domain, open source, and Creative Commons-licensed content.
It has been intentionally made very easy to incorporate new clients into the OCN. All you need to do is make an XML-RPC call advertising the content you have available and you're in.
If Bittorrent joined the OCN, it could leverage the resources of other clients in the network and give more choice to users about which software they want to use.
The eventual goal is to get this type of technology integrated directly into Mozilla, Squid, and Apache. But that will probably take some time...
When I wanted to download a Linux distro, P2P was the first place I looked. I didn't want to cost the providers a gig of traffic when they're not making any money on it. Pity I didn't really find what I was looking for. This was a while back, though.
I'm the type of guy who doesn't like sharing my bandwidth, but I'd be willing to make an exception for Open Source stuff just on the grounds the it helps alleviate the costs of hosting free stuff.
"Derp de derp."
Broadband ISP's in europe impose a maximum limit for traffic per month. How long before versions come out that only download and not upload? What open source needs is a way to make users pay with bandwith. Instead of using paypal, allow a user to pay for his download with X megs of upload capacity. This could lower costs for distro-upgrades considerably.
Also, how is this different from bittorrent?
Also, of course, neither one really fixes the Slashdot problem, because you need to set them up _before_ you're slashdotted :-) On the other hand, once you know you're in trouble, you can update your site to use them all by yourself, as opposed to bullying Slashdot into making a cache or hoping Google has one.
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
Boromir, son of Faramir, King of Gondor and Minas Tirith
Uh... Boromir is the son of Denethor, and brother of Faramir.
In the future, you may wish to take the following advice: identity theft is only effective when you just manage to get the fucking facts straight!
Eloi, Eloi, lema sabachtani?
www.fogbound.net
I looked over the website and the site for the current client, and found only faint, inspecific references to what loading such a client does to your machine and internet connection.
This is terrible.
We complain when Gator is loaded as an 'add-on' to our system, yet we don't mind if we are not allowed to download some content without loading some P2P app which then uses our disk space and internet connection to serve others?
They need to put up a specific message that says, in effect, "This download client will significantly speed up the process of obtaining this file. Once downloaded your computer will allow other people to download this same file, or portions of it, from your computer so they can gain the same speed benefit you will get. There is no security risk, and you can stop the client from letting others download this file by moving or deleting the file, or ending the client by doing x, y and z. If you wish to simply download the file normally without installing this client, click here - otherwise click 'OK'"
Yes, we all understand what P2P means - we are donating part of our computer and network to the P2P network for as long as we are connected to the internet. But this is not common terminology - ask a non-computer expert who has spent hours downloading music from their favorite P2P app what the P2P app does, and all they know is that they can get "free" music with this cool program. They often have no idea that others are downloading music from their computer, etc.
This may slow down adoption, but the reality is that the backlash that may come out against it is not worth the extra adoption it may gain without full and well-explained disclosure - as well as a method to download the file normally.
-Adam
Once you install Freenet, the link DOES work to load the Star Trek sites.
Hence, the fact that the post tells you to install the software BEFORE using the link.
Do some reading on freenet, what it does and how it works. It's quite interesting.
"Live Free or Die." Don't like it? Then keep out of the USA
I think that the CC project is a great idea and some kind of P2P distribution is great too. I do have some some minor concerns though, one is over their share alike license.
On the FSF web site there is a short list of Licenses For Works Besides Software and Documentation and the Creative Commons Share Alike license is not mentioned, so I sent the FSF an email about this and the opinion seems to be that the Share Alike license would not be considered GPL-compatible, which I think is a shame. The email about this can be found on the cc-licenses list archive
Check out MKDoc a mod_perl CMS
By the way, your home page is ugly....
Bill Stewart
New Fast-Compression-only CPR http://preview.tinyurl.com/dy575ks
You mean like a paycheck? Sorry, but free software destroyed that possibility.
Imagine content delivery system based on p2p instead of http, just try to Slashdot it! The possiblities are amazing! Now, just secure it against worms, tojan horses and altered data...
3000 dead over past 2 years, still no free Palestinians, still
The website is slashdotted.
Moderation: +4. Modded 70% Funny and 30% Overrated. 100% Saturated.
Ok, I know its off topic and all, but mod the parent up funny!
And here's a pullet surprise winning poem too accompany it:
I have a spelling checker.
It came with my PC.
It plainly marks four my revue
Mistakes I cannot sea.
I've run this peom threw it.
I'm sure your pleased two no.
Its letter perfect in it's weigh;
My checker tolled me sew.
I looked around the web site and couldn't find any mention of the relationship between this project and the original OpenContent project that maintains the Open Publication Licence. What's the story?
apparently your mod points are the same place your sense of humour is...
hey i actually like this idea. now whos gonna build the webpage to get the word out about it?
you're an idiot... thankfully you already spent all your mod points on +1 FunnyGoatseReferences, +1 InformativeInSovietRussiaPostsModerateYou, and +1 InsightfulBeowolfClusterOfPrOnServers...
One of the problems OCN faces is the seemingly obvious problem that data needs to be encoded in order to be shared. This is the problem we faced with MojoNation (the original swarm downloading system) and while throwing around ideas in a brainstorming session Bram came up with the idea of just swarming without encoding the data. This was not suitable to our needs at the time (it only works for popular, massively replicated files) but Bram stuck with the idea and developed it into BitTorrent.
The key insight here is that when data is encoded to increase its reliability within the p2p network it becomes useless to the person who is holding the data. This is not a problem for some applications, but when you are trying to solve the slahsdot effect or serve popular content it can become a limiting factor. The advantages that BT has over this system are that it does not require the data to be encoded in a special manner by the publisher and that data that is stored on the edge nodes is still useful to those nodes. A design like BT can peer data out of your browser cache and share data a larger range of data from each particular peer. This is going to be a significant advantage in the long-run.
The OCN does not require data to be encoded and uses standard HTTP for all of its data transfer.
This means that it can download content from a regular Apache web server w/o any modification whatsoever.
This also means that the peers are simply embedded web servers, can stream content (video) straight to the browser, and can use SSL out of the box.
Very cool, but what about bitrot? If I download a freebsd version from it will it be 2.2.5? Or 5.0? I hope there is a good way to check.
I hope this works. It'd be a nice alternative to porn and mp3's on P2P networks. (Even though it is its own P2P network). Maybe distributed CVS should be next? That way we don't overload a server, but we can still download just the updates we need?
I always thought P2P had fantastic potential for things like free software, educational materials, unpopular political speech...
What we need is a P2P network that outright will not allow certain types of content. That's the only way we'll ever keep the SNR down and avoid giving universities and ISPs an excuse to restrict usage.
Jim has incorrectly assumed that OCN uses the same sort of content-encoding as Justin Chapweske's previous project, SwarmCast. It doesn't: OCN strictly uses standard HTTP requests, overlaid with a series of extra headers and URI conventions that help participants find and use mirrors of the desired content.
Because the OCN's techniques use minimal deltas atop HTTP, well-documented in open specifications, it has the potential to become a standard mix-in for almost all web servers and clients... transparently bringing the power of ad-hoc replication to every web transaction.
Nice to see that yet again I got a reason to fire up that compiler.
Sorry about the writing. Robot fingers, you know? Cliff Steele in DOOM PATROL #23
Hey, I've just been through the 'pain' of installing gentoo, why the hell can't they use P2P stile downloads spanned accross the mirrors.
That 40MB file downloading at a painfull 2k because the mirror you picked is abit bussy at the moment.
And why doesn't it continue to download while compiling. P2P networks let you play a file and download at the same time.
Many more OT gripes about emerge... but I'll save them for a patch.
thank God the internet isn't a human right.
Regarding:
The World Wide Web is "the universe of network-accessible information", i.e. anything with a URI, including URIs that are not tied to a particular hostname.
The Web already includes non-location-based URIs like mid: (for referring to message-ids), and urn:sha1: for referring to a specific set of bits by their checksum.
This proposal seems like a decent way of bridging HTTP-space with URN-space, but please remember that the Web is more than just HTTP. (see also: URIs, URLs, and URNs)
Anyway, it seems to me that sites that tend to suffer from slashdotting are:
those that use dynamically-generated pages for what is basically static content: this problem can be fixed by sites making sure their content is cacheable, and further deployment of HTTP caches. (I'm not convinced a p2p-style solution is the solution here.)
those with large bandwidth needs (kernel images, linux distribution .iso's,
multimedia): as p2p software becomes more mature and widely deployed, everyone
will have a urn:sha1: resolver on their desktop (pointing to their p2p software
of choice), then whenever a new kernel is announced, the announcement can say:
Linux kernel version 2.4.20 has been released. It is available from:
2 .4.20.gz2 .4.20.tar.gz
Patch: ftp://ftp.kernel.org/pub/linux/kernel/v2.4/patch-
a.k.a. urn:sha1:OWXEOVAK2YJW3G6XSULXDWFCNWTX7B2K
Full source: ftp://ftp.kernel.org/pub/linux/kernel/v2.4/linux-
a.k.a. urn:sha1:PPWXYMA32YNDNO35UD3IQTCWBVBYK5DC
and people can just fetch the files using urn:sha1 URIs instead of everyone hitting the same set of mirrors. (gtk-gnutella already supports searching on urn:sha1: URIs)
I seem to remember an article in a LinuxJournal a while back talking about uploading the entire Debian archive onto freenet and then using apt-get to download it. If that's the case, then you just have to point it to the correct freesite and let apt-get do the rest for you. I'm sure that a similar program, if not apt, can be created to automatically download and update new opensource programs. So maybe instead of trying to reinvent the wheel, they could just do that. Plus, maybe then, there'd be a decent number of people with semi-permanant nodes on freenet.
Just my $0.02,
La Camiseta
Boromir son of Faramir is a troll. all that stuff he said is trolling. moderate accordingly.