Slashdot Mirror
Coral P2P Cache Enters Public Beta
Eloquence writes "infoAnarchy reports that Coral, a peer-to-peer webcaching system, has gone into public beta. Currently the Coral node network is hosted on Planet-Lab, a large scale distributed research network of 400 servers. You can use Coral right now by appending "nyud.net:8090" to a hostname. View Slashdot through Coral. Is this the end of the Slashdot effect?"
Just kidding.
Google cache has been a good helper to me for some time.
So this is not so new to me regarding slashdot effects.
Why would we want to view it through the cahce, we couldn't see the nice news story you just posted. However.. I am supprised how recently it is cached.
Notice that because of the caching system it isn't fully current...
Of, well, slashdoting the solution to slashdotting? Really cool idea though. Nice!
except the cache doesn't contain this post...yet
I hope this isn't the end of the /. effect! What would we do w/o webservers crashing under tremendous loads?!? WE NEED the /. effect! I hope this technology crashes and burns...
Then again it might not be so bad....
"The object of war is not to die for your country, but to make the other bastard die for his." - Patton
so it's like this... people click on a link on slashdot, which gets farmed out to the p2p network to get the cached copy, but there's so many people clicking the link to get the cached copy that they are only slashdotting their own computers since they are all part of the p2p network too! now we can all collectively feel the slashdot effect!
oh, first post?
You should never take life too seriously - You'll never get out of it alive.
you can ensure that your readers can still access a certain web page or files, when the multitude of readers would otherwise overload the website and make the content unavailable.
well apparently all html content, including files, will be cached. this is a great way to get around downloading from snail-pace sites, (although i will be checking md5sums)
Marge, get me your address book, 4 beers, and my conversation hat.
http://www.scs.cs.nyu.edu.nyud.net:8090/coral/ if we do /. it...
While their system would be pretty good (supposing it can withstand a slashdotting) for cacheing large files, it's not very useful for websites. Websites usually have lots of additional images, links, and whatnot, and as is currently, the system doesn't rewrite URLs.
Hmm..if a web contains a META refresh tag redirection then the cache looses it's point becouse you're redirected to the original location not a cache :(
In case Coral gets slashdotted, use this mirror to view slashdot
!
^_^
shame the cache is not up-to-date.
Is there any way to force firefox to append the p2p information automatically so we browse the cache all the time instead of the normal websites?
"Is this the end of the Slashdot effect?"
./ colo guys at exodus would love for you to run one :).
haha no - only the lateral shifting of the slashdot effect to your local lan as some dope sets up a cache server in your office. Im sure the
as will ISPs if it takes off. Right now with bandwidth usage centralized it's pretty easy to bill for it. If you decentralize it with p2p via millions of always on unmetered clients/servers it gets hard, if not impossible. I kinda hope it doesn't take off, since if it does it could end unmetered Internet access...
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
This would also by pass any restricted sites your company may be blocking...
No, after the FBI has a gander at the servers, and puts them in a truck and drives off, the Slashdot effect will be alive and well.
"Who are in control, they are not in control of anything - they don't even control themselves!" - Glen Beck
Oh great, the /. effect hits the Coral reefs. Environment, watch out.
There has been something called FreeCache for some time already. It helps with big files (over 5MB). But of course, why use it when you can fuck people's servers instead? Sometimes it is more about will than technology.
This is a Content Distributon network of cooperating servers colloborating to exchange information and 'level out' excess demand by distributing reqiests among n servers. Like Akamai's EdgeSuite. based on a quick read of the front page. The providors of content in their network are never the consumers if content. thus i don't know why they call it peer-to-peer? anyone?
Some friends and I have an approx 10 MBytes application we want to distribute over the Internet, looking into hosting costs we see that it would cost us a bundle. So does coral let us serve our file to a slashdot-like crowd without breaking the bank?
Some reason, this works but this doesn't... guess there are limits to recursion. If for some reason the last link works, keep adding nyud.nets...
It's turtles all the way down...
I love the idea. I hope that it doesn't have as much trouble being functional as projects like freenet.
http://www.archive.org/web/freecache.php
It isn't P2P web proxy, it's just "big pipe"-based distributed one. Supposedly a great way to prevent slashdoting (just use http://freecache.org/http://mytinysite.com instead of http://mytinysite.com and everything goes from the cache, tiny site receiving only header requests to chceck if the document hasn't changed in the meantime) it's hardly known, way too quiet as for a project that useful. P2P may be faster and cheaper but certainly less reliable...
45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
http://slashdot.org.nyud.net:8090/ caches only the /. homepage. Doesn't it analyze hyperlinks?
If the color hurts your eyes maybe you are spending TO MUCH TIME at slashdot.
Many times it seems a bittorrent tracker is down due to bandwidth issues. If I "corralized" it...could this alleiviate the problem?
druken college kid: "dude, we should make a program that downloads and stores the internet"
druken buddy: "dude, after i piss, i'm so there."
i wonder if google will index this, and cache it? wow that could make for some sweet folly the next time giant company x accidentally posts that they knowingly discriminate against minorities or hate gay midgets or fixed nazi computers that kept track of concentration camp prisoners.
holy crap did that get offtopic.
scott king
I was playing around with this the other day. I tried it out with my filerush.com page which has the main html, but all images are loaded off a second images.domain.com server. Coral will get the main server's files that match the origination URL that you passed it, but will skip all files that are sitting off another server. So basically it'll handle all relative links, but no hard links to other sites. It would be killer if they somehow made it configureable in your browser's proxy settings.
Just what we need, a new window of opportunity for goatse trolls.
Go here: http://www.newberrycollege.net/
Notice the random question at the bottom of the page.
Then go here: http://www.newberrycollege.net.nyud.net:8090/
The question is not randomly generating. They should have some checking for this such that if the data varies by the second, that it does not cache the page or invisible frames the HTML and filter the content it can cache.
Bleh.
It will work in certain cases, but generally I am not that happy.
http://www.scs.cs.nyu.edu/coral/stats/
http://fuckIT.slashdot.org/it/04/08/28/2330252.sht ml?tid=95&tid=218
Goatse-links trolls will be back, with slashcode showing the same domain for every link, I think CmdrTaco has some work to do now.
The IT section color scheme sucks.
there's a reason it hasn't been released yet.
To save their bandwidth, you should've linked to their mirror!
http://www.nyud.net.nyud.net:8090
Beware: In C++, your friends can see your privates!
This system fails because most commercial sites, and many others, will lose the ability to track web usage for site tuning and marketing response. Sites will be built -- if need be -- with specific settings or configurations to confound the coralling of their pages.
Its a noble goal, but ultimately will go the way of the video phone -- which apart from conferences planned in advance, remains a novelty dispite perfectly adaquate technology -- nobody wants a suprise video call because nobody wants to be a 50's housewife who's self esteem is tied to the cleanliness of their floors and their ability to have perfect hair and a matching necklace and top all the time "in case someone calls".
If people don't want it, it will fail regardless of how well done.
--
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
The problem is that it doesn't seem to be compatible with Microsoft DNS severs. Below is a copy of the DNS log when I issue a query here, on my LAN which has a Microsoft DNS server running on Windows 2000, which then forwards through the University of Wisconsin. You can see that at the end it says "The DNS server encountered an invalid domain name." Perhaps someone who knows more about DNS can tell where the problem is?
Would this work for pr0n sites and make the content free? Yeah, Yeah, I know you can get free pr0n but it is the paysites I am talking about...
Friends help you move...
REAL Friends help you move dead bodies... ^_^
We all hate the new color scheme.
Pretty picture :)
:(
http://www.scs.cs.nyu.edu/coral/stats/
Doesn't give a usable time scale though; it has "HTTP requests", but not "per second" / "per minute" or anything
I mod down anyone who says "I will be modded down for this", regardless of the rest of their comment
Color harmony
...bittorrent trackers are applications, not static webpages. If your tracker goes down, try to find another one.
-ReK
md5sum -c reality.md5
reality: FAILED
md5sum: WARNING: 1 of 1 computed checksum did NOT match
Although I agree with others, it doesn't really compare to FreeCache. I still wonder why that never got much attention. It's an insanely great idea. Ah well. Between that, Corla, and BitTorrent, you never have to worry about /.'ing again when you submit your tiny personal site.
In other news (for the morons who continue posting and whining), you can still remove the it prefix from the /. URL, removing the fugly colour scheme. And there was much rejoicing in the land.
(-:Stephonovich:-)
"Who needs reincarnation when we've got parallel universes?" -Me
No.
Safari can't open the page "http://slashdot.org.nyud.net:8090/" because it could not connect to the server "slashdot.org.nyud.net".
I believe there is a term for this.
'doh!'
Yeah, I knew some one would do this .........
* Carthago Delenda Est *
I've tried http://slashdot.org.nyud.net:8090/ in several different browsers, and the page only brings up gibberish. Just like when you hit View Source on a jpeg or some binary file. What's the deal?
I don't know about anyone else, but when I viewed slashdot through the cache, my login was no longer in effect.
If this is a consistent problem across the board for page caching, I can't see it being useful for sites that use logins. And I'll be damned if I'm going to use the login link in the cached page.
If there's anything more important than my ego around here, I want it caught and shot immediately.
http://shit.slashdot.org/article.pl?sid=04/08/28/2 330252&threshold=-1&tid=95&tid=218
Ah, now I can see.
Show me on the doll where his noodly appendage touched you.
This would be called a "good and useful feature to a more secure and functional internet".
Well, if coral really worked, it would support
the cutting edge browsers, like lynx. I just
a string of gibberish when I tried it.
----- Why sig when you can sign? PGP key id 7675D05E
I believe there are couple of problems with this scheme.
1. The site owner will not be able to monitor hits on the site.
2. This appears to work for static pages but it won't work with dynamic pages ie. php.
I have to believe that ISPs that provide web services would find their revenue reduced since they would not see all the hits on the site.
Plus as others have said Google doesn't convert links.
I haven't checked the terms of use to see if I'm allowed to use this for my work web site, though maybe with a cash or hardware donation, or by running a high-bandwidth node, I can get permission.
What I'm thinking is that at work I run a multi-server site that gets massively bogged down for short periods when it tries to handle upwards of 35,000 concurrent sessions. Bandwidth is not the problem, the application is, and it can't be rewritten for reasons that piss me off and I have no budget for more servers and no management support to run a static cached version of the site.
So I was wondering if it was possible to have the site automatically direct visitors to the Coralized URL when the site load gets too high. Either a manual change or an automatic one would be ok. I have some ideas on how this could be done using a failover server config on our ServerIron. Possibly a router config can also do this, though we don't run our own router since it's at a colocation facility. Worst case scenario is I can edit the home page to redirect to Coral when the load gets high.
Are there any other Slashdotters looking to use Coral in similar ways? If you have any ideas to share I'd be all ears.
The global economy is a great thing until you feel it locally.
imagine if we all used our max upload bandwidth 24/hrs a day. ISP would need to modify their networks to work around this. At least I assume they would. As it is, many 'unmetered' isps will start sending you nastygrams if you make heavy use of your upload bandwidth, but otherwise look the other way when you run a server. Keep in mind that all these p2p apps violate most IPS' TOS (mine doesn't let you run a server of any kind, and while there are places where enforcement of that would be silly, there's still plenty of room for a crack down).
Hi! I make Firefox Plug-ins. Check 'em out @ https://addons.mozilla.org/en-US/firefox/addon/youtube-mp3-podcaster/
Although I can browse both slashdot and SomethingAwful forums through Coral, I cannot do so while logged in. I wonder if the "user is logged in " logic is confused.
Links (another text browser) has no issue negotiating compression, but their cache forces it/forgets adding the proper headers.
wow good point - this thing recurses. It slows down the more you ask it to recurse.. http://www.nyud.net.nyud.net.nyud.net.nyud.net.nyu d.net.nyud.net.nyud.net:8090/
:)
cool
Not yet!
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
So it looks like this thing is fairly useless for FP trolls like me. It trails by at least an hour in the caching.
as far as content delivery? anybody explain the difference?
Just in case the coral site goes down... here's a link to the coral cache:
http://www.scs.cs.nyu.edu.nyud.net:8090/coral/
God became man to enable men to become sons of God. -C.S. Lewis
you can still remove the it prefix from the /. URL, removing the fugly colour scheme
Thank-you, thank-you, thank-you, thank-you.
Try something like:e t.nyud.net.nyud.net.nyud.net.nyud.net.nyud.net.nyu d.net.nyud.net.nyud.net:8090/
http://nyud.net.nyud.net.nyud.net.nyud.net.nyud.n
Let's just re-route all of the traffic on the internet through a P2P cache site. Doesn't this kind of defeat the purpose by overloading the service?
Mod points are pointless when you browse at -1.
Check out their logs...
...note the recent blip?
Coral Statistics
Maw! Fire up the karma burner!
Is this the end of the Slashdot effect?
/. from the cache, I'd say "no".
Considering that it took 3x longer to load
Could people in China use it? Or could I use it to watch the olympics online if I live in the US?
http://www.google.com.nyud.net:8090/This will be interesting as Google chooses language by IP range. Now to workaround the persistant Russian versian we will all need to install the famous exp 2038 cookie. No seriously - do you want a local copy of kiddie porn served from your boxen?
Open to abuse
Is this the end of the Slashdot effect?" haha no - only the lateral shifting of the slashdot effect to your local lan as some dope sets up a cache server in your office.
This is what coral says:
One of Coral's key goals is to avoid ever creating hot spots that might dissuade volunteers from running the software for fear of load spikes. It achieves this through a novel indexing abstraction we introduce called a distributed sloppy hash table (DSHT), and it creates self-organizing clusters of nodes that fetch information from each other to avoid communicating with more distant or heavily-loaded servers.
DDoS meets, distributed service and there's more of us than there are of you. Thanks for playing but the game is over. As thier effectiveness wanes, look for Bill to cut such things from Microsoft's expenses or to shift your workload to some other disruptive behavior which will again be defeated.
Friends don't help friends install M$ junk.
Friends don't help friends install M$ junk.
http://google.com.nyud.net:8090/
Will show a dynamic language. Accept XOR this or the persistant GOO cookie.
And I'm sure there will be goatse trolls - CowboyNeal will need to change the slashcode
Nice.. except it doesn't actually work (anymore).
The question is not randomly generating. They should have some checking for this
There's no reason newberrycollege.net couldn't have implemented HTTP cache control measures.
What would be interesting is if they made a browser plugin that would cache and share content. This would actually work better if it was server-side, and had clients automatically redirected to somebody who recently downloaded the page and is running the sharing plugin if the server is heavily loaded. In this case, the server would work rather like a BitTorrent tracker. Instant mirroring of highly trafficed sites, and doesn't require the user to install a plugin.
And the l33t shall inherit the 34r7h.
its just that more people need to use it
I looked up http://cnn.com.nyud.net:8090 and was shown the news from September 12th, 2001.
about.com and google are just two sites that it's easy to bounce an arbitrary redirect through.
167 posts and no 5 moderated IT Slashdot color scheme jokes? Amazing.
Here you go: Click Here
So the best way to test it is for Slashdot to add a user preference to append nyud.net to every inline href, so that if you turn that on you don't have to do it manually.
Well, it does and it doesn't. If you hit www.fyr.com.nyud.net:8090 and click Random, you get the first image. But after that... same image, over and over again. You have to explicitly select another image, or forget it.
I am using proxomitron (nifty, just for reference) to rewrite all URLs to add the .nyud.net:8090 on the end there.
I imagine similar things could be done on a router or proxy for an institution. Interesting idea, in general.
Uh oh:
n lo ad/1/6/5/165b076b-aaa9-443d-84f0-73cf11fdcdf8/Wind owsXP-KB835935-SP2-ENU.exe
http://download.microsoft.com.nyud.net:8090/dow
Here's something I've often wondered - what's the go here? Microsoft don't want anyone mirroring Windows XP Service Pack 2 (despite lots of places still having it mirrored), but what about proxies?
The practice of allowing portnumbers seems dangerous. I can imagine links like http://localhost.19.nyud.net:8090/ or http://loghost.515.nyud.net:8090/ being used for nefarious purposes.
Regards,
--
*Art
but how do i become a contributing member?
i run a site, but i still have some extra bandwisth, and this seems like a damn fine cause.
If you browse through the p2p cache, it also hides your IP address.
Check this out: Normal browsing and Coral browsing.
Could that be useful against some DDOS?
Not all of course, because you need access to back-end for sites. But for some sites that provide general info, could that make them immune?
Other question. If the main site disappears. How long will its info stay cached?
I like my porn, but this seems like it would be too easy to abuse by spammers and cheap porn operators.
We use these types of records to aid in redirecting resolvers to nearby Coral proxies
I am not a network expert, but AFAIK geographical 'nearby' does not neccessarily mean 'short latency'. For instance when I access a server which is located in the same street where I live, traffic is routed through another city.
Aren't routing protocols designed to choose 'nearby' paths in terms of fast transfer and low latency regardless of location?
Cool project nonetheless. Kudos.
-silence
Dyslectics of the world, untie!
This is great for my Gallery of photos. I won't post a link (it'll still get slashdotted!), but what it does is it mirrors a photo when the first user clicks on it, and then from then on, there is no load on my server running behind a DSL line. This is perfect for mirroring my photos...it all gets handled automatically ! and it saves me a ton of bandwidth !
And if the links are absolute...how about a Firefox plugin that 1) notices you're looking at a Coral cache, and 2) rewrites any links pointing at same domain to go to the Coral cache...
Dunno.. lets find out....
Everyone get your free pr0n here! >> http://www.scs.cs.nyu.edu
Join the Slashcott! Feb 10 thru Feb 17!
Of course, slashdot wants to fight me making this comment. Oh well, I can't take a hint. Someone can mod me into oblivion if they want.
The problem with quotes on the internet, is that nobody bothers to check their veracity. -- Abraham Lincoln
This will only help with bandwidth problems.
If you have bandwidth, run a caching proxy (something like Squid) between the application and the network. If several of the requests you serve are getting the same content, then it won't even need to hit the application.
Now, if 100% of your pages served are unique, nothing in the world can help you, other than replicating that application on more or faster servers.
Slashdot gets worse every day... Pipedot: News for nerds, without the corporate slant
Is dealing with so many concurrent connections the problem? Are there elements you can cache? Then, you might look into using Squid in reverse proxy mode. Users connect to Squid, which connects to your servers, getting the page as quickly as possible, serving to the client at the appropriate speed. If you let it cache, then it can serve requests without hitting the servers.
Using Coral would mess up dynamic content, unless you avoid using cookies. For static or dynamic content, I don't see Coral doing anything more than Squid would, unless bandwidth is an issue.
I thought I was being clever/helpful by writing a little script/addin to IE that allowed you to jump to the coral cache of that page.
. htm
Details are here: http://www.cbeach.pwp.blueyonder.co.uk/coraladdin
I then realized that if you could get to the page then you wouldn't need the script and if you couldn't get to the page then the script didn't work!
Dunno why I'm even posting this tbh...
----- I refuse to have an argument with an unarmed person
This will only help with bandwidth problems.
Sounds like he has political problems as well. He might be able to get this through management.
Now, if 100% of your pages served are unique, nothing in the world can help you, other than replicating that application on more or faster servers.
A caching proxy will still improve the performance of your static objects (images, stylesheets). This will then relieve pressure on your dynamic engine. This is similar to having a dedicated server handling static objects without the added complexity.
You meant, append ".nyud.net:8090" to a host name. Note the leading period after the first quote above. As written, the story would have you do this:
...which obviously isn't correct. At list the "View Slashdo through Coral" link is correct as follows:
http://slashdot.orgnyud.net:8090/
http://slashdot.org.nyud.net:8090/