Slashdot Mirror

← Back to Stories (view on slashdot.org)

Coral P2P Cache Enters Public Beta

Posted by timothy on from the will-it-scale dept.

Eloquence writes "infoAnarchy reports that Coral, a peer-to-peer webcaching system, has gone into public beta. Currently the Coral node network is hosted on Planet-Lab, a large scale distributed research network of 400 servers. You can use Coral right now by appending "nyud.net:8090" to a hostname. View Slashdot through Coral. Is this the end of the Slashdot effect?"

24 of 254 comments (clear)

  1. Google by asd-Strom · · Score: 3, Informative

    Google cache has been a good helper to me for some time.
    So this is not so new to me regarding slashdot effects.

    1. Re:Google by Gherald · · Score: 2, Informative

      >Are you saying with Coral that you can?

      With Coral you can get it cached just by asking for it. Of course, the Coral pcs have to connect to it at least once.

      You cannot get google to cache a page at your request -- no matter how hard you try :)

      --
      The unofficial /. digest
    2. Re:Google by bogie · · Score: 4, Informative

      Google cache tip for you. There is a bookmarklet for Firefox where you simply click the bookmarklet and Google's cache of the page opens up. Its a nice feature to have at your fingertips. You can get the code at the very bottom of the following page, just drag it to your personal toolbar.

      http://www.rentzsch.com/notes/googleCacheHacking

      If the page won't load at all thus negating the above just use the following example to load a page.

      http://google.com/search?q=cache:www.slashdot.or g

      --
      If you wanna get rich, you know that payback is a bitch
    3. Re:Google by doofsmack · · Score: 5, Informative

      Talking about bookmarklets, I just wrote a quick little bookmarklet to redirect you to the Coral cache of the current page. Here it is:

      javascript:location.href=location.href.replace(/ht tp\:\/\/([a-zA-Z\.]+)\/(.*)/, "http://$1.nyud.net:8090/$2");void(0)

      And if slashdot's tendency to insert spaces in long strings screws that up, try grabbing it from here

  2. Delay by Anonymous Coward · · Score: 1, Informative

    Notice that because of the caching system it isn't fully current...

  3. Re:Slashdotted already by Anonymous Coward · · Score: 2, Informative
    No, it really is down, at least from here:

    Windows:
    C:\>ping slashdot.org.nyud.net
    Ping request could not find host slashdot.org.nyud.net. Please check the name and try again.
    Linux:
    ~$ping slashdot.org.nyud.net
    ping: unknown host slashdot.org.nyud.net

    ~$dig slashdot.org.nyud.net
    ; <<>> DiG 9.2.1 <<>> slashdot.org.nyud.net
    ;; global options: printcmd
    ;; connection timed out; no servers could be reached
    Seems their nameservers have some kind of problem. I am in the Midwest, going through an AT&T OC3 (everything else works fine from here; it's not a local problem). It works OK when I check from our California-based servers that peer with Mae West, however.
  4. Re:Not too good for websites by Coneasfast · · Score: 4, Informative

    links should be (and usually are) relative, eg:

    img src="img/logo.png"
    not:
    img src="http://slashdot.org/img/logo.png"

    or whatever so this shouldn't be a problem

    --
    Marge, get me your address book, 4 beers, and my conversation hat.
  5. What about Freecache? by SharpFang · · Score: 2, Informative

    http://www.archive.org/web/freecache.php

    It isn't P2P web proxy, it's just "big pipe"-based distributed one. Supposedly a great way to prevent slashdoting (just use http://freecache.org/http://mytinysite.com instead of http://mytinysite.com and everything goes from the cache, tiny site receiving only header requests to chceck if the document hasn't changed in the meantime) it's hardly known, way too quiet as for a project that useful. P2P may be faster and cheaper but certainly less reliable...

    --
    45 5F E1 04 22 CA 29 C4 93 3F 95 05 2B 79 2A B2
  6. Re:Usefulness by chewy_2000 · · Score: 2, Informative
    Looks like it. Haven't tried it personally yet, but I will.

    Here.

  7. Stats! Slashdot has it REALLY working! by Danathar · · Score: 4, Informative

    http://www.scs.cs.nyu.edu/coral/stats/

  8. Re:Only the top page? by mothz · · Score: 4, Informative

    Doesn't it analyze hyperlinks?

    All the links on Slashdot have the format
    <a href="//slashdot.org/blahblahblah">
    so that they will always link back to Slashdot. Most websites just use "blahblahblah" or "/blahblahblah" for their links. For example, links on google.com.nyud.net are fully functional.

  9. "Invalid domain name in packet" by Anonymous Coward · · Score: 5, Informative
    It really DOESN'T work for a lot of people.

    The problem is that it doesn't seem to be compatible with Microsoft DNS severs. Below is a copy of the DNS log when I issue a query here, on my LAN which has a Microsoft DNS server running on Windows 2000, which then forwards through the University of Wisconsin. You can see that at the end it says "The DNS server encountered an invalid domain name." Perhaps someone who knows more about DNS can tell where the problem is?

    Rcv 10.76.0.2 0004 Q [0001 D NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
    UDP question info at 014D5A0C
    Socket = 384
    Remote addr 10.76.0.2, port 1263
    Time Query=4338128, Queued=0, Expire=0
    Buf length = 0x0200 (512)
    Msg length = 0x0027 (39)
    Message:
    XID 0x0004
    Flags 0x0100 QR 0 (question) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 0 Z 0 RCODE 0 (NOERROR)
    QCOUNT 0x1 ACOUNT 0x0 NSCOUNT 0x0 ARCOUNT 0x0
    Offset = 0x000c, RR count = 0
    Name "(8)slashdot(3)org(4)nyud(3)net(0)"
    QTYPE A (1)
    QCLASS 1
    ANSWER SECTION:
    AUTHORITY SECTION:
    ADDITIONAL SECTION:

    Snd 144.92.254.254 39b0 Q [0001 D NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
    UDP question info at 0109200C
    Socket = 408
    Remote addr 144.92.254.254, port 53
    Time Query=0, Queued=0, Expire=0
    Buf length = 0x0200 (512)
    Msg length = 0x0027 (39)
    Message:
    XID 0x39b0
    Flags 0x0100 QR 0 (question) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 0 Z 0 RCODE 0 (NOERROR)
    QCOUNT 0x1 ACOUNT 0x0 NSCOUNT 0x0 ARCOUNT 0x0
    Offset = 0x000c, RR count = 0
    Name "(8)slashdot(3)org(4)nyud(3)net(0)"
    QTYPE A (1)
    QCLASS 1
    ANSWER SECTION:
    AUTHORITY SECTION:
    ADDITIONAL SECTION:

    Rcv 144.92.254.254 39b0 R Q [8081 DR NOERROR] (8)slashdot(3)org(4)nyud(3)net(0)
    UDP response info at 012DB8AC
    Socket = 408
    Remote addr 144.92.254.254, port 53
    Time Query=4338128, Queued=0, Expire=0
    Buf length = 0x0200 (512)
    Msg length = 0x00e0 (224)
    Message:
    XID 0x39b0
    Flags 0x8180 QR 1 (response) OPCODE 0 (QUERY) AA 0 TC 0 RD 1 RA 1 Z 0 RCODE 0 (NOERROR)
    QCOUNT 0x1 ACOUNT 0x4 NSCOUNT 0x2 ARCOUNT 0x2
    Offset = 0x000c, RR count = 0
    Name "(8)slashdot(3)org(4)nyud(3)net(0)"
    QTYPE A (1)
    QCLASS 1
    ANSWER SECTION:
    Offset = 0x0027, RR count = 0
    Name "[C019](4)nyud(3)net(0)"
    TYPE 39 (39) CLASS 1 TTL 1333 DLEN 25
    DATA Unknown resource record type 39 at 012DBC41.
    Offset = 0x004c, RR count = 1
    Name "[C00C](8)slashdot(3)org(4)nyud(3)net(0)"
    TYPE CNAME (5)
    CLASS 1 TTL 0 DLEN 15
    DATA (8)slashdot(3)org[C033](4)http(2)l2(2)l1(2)l0(5)n yucd(3)net(0)
    Offset = 0x0067, RR count = 2
    Name "[C058](8)slashdot(3)org[C033](4)http(2)l2(2)l1(2) l0(5)nyucd(3)net(0)"
    TYPE CNAME (5)
    CLASS 1 TTL 1335 DLEN 2
    DATA [C033](4)http(2)l2(2)l1(2)l0(5)nyucd(3)net(0)
    &am p;n bsp; Offset = 0x0075, RR count = 3
    Name "[C033](4)http(2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
    TYPE A (1)
    CLASS 1 TTL 60 DLEN 4
    DATA 139.91.70.71
    AUTHORITY SECTION:
    Offset = 0x0085, RR count = 0
    Name "[C038](2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
    TYPE NS (2)
    CLASS 1 TTL 1991 DLEN 19
    DATA (3)139(2)91(2)70(2)71(3)ip4[C041](5)nyucd(3)net(0 )
    Offset = 0x00a4, RR count = 1
    Name "[C038](2)l2(2)l1(2)l0(5)nyucd(3)net(0)"
    TYPE NS (2)
    CLASS 1 TTL 1991 DLEN 16
    DATA (3)141(3)213(1)4(3)202[C09E](3)ip4[C041](5)nyucd( 3)net(0)
    ADDITIONAL SECTION:
    Offset = 0x00c0, RR count = 0
    Name "[C091](3)139(2)91(2)70(2)71(3)ip4[C041](5)nyucd(3 )net(0)"
    TYPE A (1)
    CLASS 1 TTL 603196 DLEN 4
    DATA

    1. Re:"Invalid domain name in packet" by mfreed · · Score: 5, Informative
      It appears that the Windows 2000 DNS server you are using is not aware of DNAME records (RFC 2672):

      Name "[C019](4)nyud(3)net(0)"
      TYPE 39 (39) CLASS 1 TTL 1333 DLEN 25
      DATA Unknown resource record type 39 at 012DBC41.
      We use these types of records to aid in redirecting resolvers to nearby Coral proxies (by mapping nyud.net to a "hierarchical" name http.l2.l1.l0.nyucd.net. The goal is that once you find a "nearby" server, you should remain nearby.

      Given that the DNAME RFC is from 1999, it appears that some old DNS servers do not handle this record type well. We'll look into some alternatives or work-arounds. (Perhaps you can contact me directly to see if subsequent changes can fix your problem.)

      Thanks for the detailed report!
      --mike

    2. Re:"Invalid domain name in packet" by Anonymous Coward · · Score: 1, Informative

      No, that doesn't work. For example:

      http://www.slashdot.org.http.l2.l1.l0.nyucd.net: 8090/

      It resolves to the correct machine, yes, but the proxy doesn't understand how to extract the machine's hostname from a hostname of that form, so the HTTP request fails.

  10. Re:me thinks not P2P by Inominate · · Score: 4, Informative

    It's not p2p.

    It's 'distributed'.

    Peer to peer implies that the users of the service are the ones supporting it's existance.

  11. Re:Only the top page? by SirDaShadow · · Score: 2, Informative

    so that they will always link back to Slashdot. Most websites just use "blahblahblah" or "/blahblahblah" for their links. For example, links on

    Well...all they have to do is have some modifying code like CGI-Proxy does....

  12. Not a good solution by pyrrhonist · · Score: 2, Informative
    From the FAQ:
    What files are being served by FreeCache?

    FreeCache can only serve files that are on a web site. If the link to a file on that web site goes away, so will the file in the FreeCaches. Also, there is a minimum size requirement. We don't bother with files smaller than 5MB, as the saved bandwidth does not outweight the protocol overhead in those cases.

    --
    Show me on the doll where his noodly appendage touched you.
  13. Re:Hosting companies'll hate this.... by interiot · · Score: 2, Informative
    Erm? If you mean the "5mb webspace" places, no, they don't charge for bandwidth. They just cancel your account if you post porn or anything else that will get heavily downloaded.

    As far as I know, anybody in the 0.5gig/month or over (all the way up to the backbone carrierers, which have to have peering agreements as an exception to the rule of charging for bandwidth) charges per megabyte.

  14. Re:Also a proxy... by interiot · · Score: 4, Informative
    There are actually a lot of sites out there that will let you access arbitrary content from elsewhere. Most corporate restricting proxies will block at least some of them (but it's impossible to get all of them). So something that could be as high-profile as Coral is less useful compared to some of the more obtuse of these:
    • google cache (this has been periodically blocked at my company)
    • the internet archive
    • online translation sites (eg. if it's an english site, have the translator go from japanese to english... none of the words will be recognized as japanese, so it will pass them all as-is)
    • several others I'm forgetting at the moment...
  15. Re:Incompatible with logged in browsing by maskedbishounen · · Score: 5, Informative

    Sounds like we need a little lesson on How cookies work.

    To summarize it, though, they're set on a per-domain basis.

    www.apple.com can set a cookie.
    store.apple.com can set a cookie.

    The two cannot interact with each other; however, .apple.com can interact with any Apple subdomain.

    microsoft.com cannot access any of your apple.com cookies.

    Thus, nyud.net cannot access your /., or any other cookies.

    --
    "An infinite number of monkeys typing into GNU emacs would never make a good program."
  16. Re:Is this the solution? by focitrixilous+P · · Score: 2, Informative
    Bittorrent is your friend. It's as common as AIM or IRC these days, instead of pulling the whole file from a central server, only the first few need to use a server host, and everyone else shares with each other. Most big linux distros do it with 650 MB files, or for large video files. No reason it wouldn't work for you.

    Here, I'll even link you to a good client that will give you a nice GUI for starting out. Another Bittorent Client for all OSes.

    --
    SAILING MISHAP
  17. Not quite, but here is what /. looks like! by Tailhook · · Score: 2, Informative

    Check out their logs...

    Coral Statistics

    ...note the recent blip?

    --
    Maw! Fire up the karma burner!
  18. Re:Possible problems with this scheme... by geminidomino · · Score: 2, Informative

    I have to believe that ISPs that provide web services would find their revenue reduced since they would not see all the hits on the site.

    It seems you're confusing a "cache" with a "proxy." A "cache" is only DESIGNED to work on static pages, and it doesn't hit the page more than once (barring refreshing). That's what "cache" means. The pages are stored on the cache server and fed to the clients as they get requested, cutting down on hits to the actual site.

  19. Re:What would make it p2p by rob101 · · Score: 2, Informative

    The above link from an anonymous coward points to a paper through some weird obfuscation that does just that. HTTP(P2P)