Slashdot Mirror


AOL Blocks Links from LiveJournal

Evan Martin writes "LiveJournal.com is an open-source weblog site with over a million users, some of whom use AOL. Last week, AOL began blocking all HTTP requests with "www.livejournal.com" Referer headers. This is a common practice by image hosting sites to prevent off-site linking of their images and 'bandwidth theft'. However, in AOL's case, they're blocking everything, not just images, effectively breaking all links to any AOL member's site--but only from LiveJournal. To be clear: nobody on LiveJournal can even make a link to any AOL member site without getting a '404 Not Found' error. We've also heard reports of the same thing happening on AOL properties (Netscape, Compuserve). This concerns us because we have to deal with the support requests: it worked in the past for our users, and it continues to work for other sites, so our users think it's our fault."

Martin continues: "We've tried to contact AOL three different ways, all without success. We've also told our users to contact their tech support. At one point, an AOL staffer pointed out that FTP access still worked (which is probably because FTP has no "Referrer" concept), and so, as an interim fix, we're rewriting all HTTP URLs to use FTP on the AOL properties where that works instead. This means that users can again host their images on the AOL webspace they're paying for, but more importantly, it means they can simply link to their webpage.

We wouldn't be so upset if they were simply blocking images. Bandwidth use is a valid concern, after all, and we even provide step-by-step instructions for people to configure their webservers to prevent image "theft". However, because they're blocking all access, including regular links, this looks like it's either a mistake, or something more insidious (the conspiracy theorists have pointed out that AOL has just launched their own competing weblog product, also based on "journals").

Although CI Host sued AOL recently for being blocked, we really don't want to do that. We still suspect that this was all just a mistake, and hopefully, by making this public, we'll manage to get their attention, since all our previous attempts have failed."

24 of 396 comments (clear)

  1. F12 by Anonymous Coward · · Score: 3, Informative

    Enable referrer logging

    1. Re:F12 by Nasarius · · Score: 5, Informative

      Gotta love Opera :)

      --
      LOAD "SIG",8,1
  2. Use link referers by EDA+Wizard · · Score: 4, Informative
    Until this mess gets sorted out, people should use a free link relocator service. Make A Shorter Link would work well for links to AOL pages.

    It wouldn't help people with embedded links to images at AOL, but at least it could get people to AOL without any additional clicking.

  3. Solution: by WCMI92 · · Score: 3, Informative

    Don't use an ISP that is "broken". AOL has little to recommend it.

    I use Adelphia PowerLink at home. On the road, I have a dial up account with a local ISP with dial up numbers in the cities I frequently have to visit.

    --
    Corporatism != Free Market
  4. Quick fix for HREFs viewed by MSIE by inertia187 · · Score: 5, Informative
    That's wrong of AOL, but if you're a LiveJournal user in a bind and really want to fix the links (but nothing else) fast, here's a JavaScript that you can load in all of your pages. You just need to load it once, and the page will work.

    Unfortunately, this trick really only works with MSIE. But it's better than nothing.
    <script language="javascript" type="text/javascript" src="http://www.martin-studio.com/js_tools/strip_r eferrer.js"></script>
    The above should all be on one line. Check for extra white space where the line feed got placed by Slashdot's bug (thanks alot).

    It should be strip_referrer.js with no space. Why does Slashdot do that??
    --
    A programmer is a machine for converting coffee into code.
    1. Re:Quick fix for HREFs viewed by MSIE by Mark+J+Tilford · · Score: 2, Informative

      I believe the reason for that bug is as follows:

      If a page has a single line which is wider than the browser windows, then every line on that page will expand to that width, which forces the reader to scroll horizontally and back every single line, which is very annoying.

      Slashdot avoids this by simply adding a space every N characters. This could be better done by having it add a space if there are N consecutive nonspace characters, but it is done for good reason.

      --
      -----------
      100% pure freak
    2. Re:Quick fix for HREFs viewed by MSIE by Anonymous Coward · · Score: 1, Informative

      You're correct. They don't allow JavaScript in entries/styles because it can be used in various ways that open up a gaping security hole. Another method for getting around this obnoxious thing will have to be found. (not that I have this problem, as I use Opera and don't send Referrer headers anyways...)

      Bob Calise
      bobert225.livejournal.com

  5. Re:tinyurl? by Jouster · · Score: 3, Informative

    TinyURL uses a Location: header, which should kill off the referer, yes. But asking everyone to TinyURL their images is a bit much, don't you think? Besides, some browsers don't like having 3XX statuses (stati?) as replies to their image requests, so you'd break some people.

    AOL just needs to stop doing that shit. Clamp down on the people transferring 200 gigs in the exhibitionism-community-of-the-week, and leave everyone else alone.

    Jouster (My LJ)

  6. Re:tinyurl? by Chmarr · · Score: 4, Informative

    It TOTALLY depends on the browser you're using.

    If you're on web page A, click on a link to B and it redirects to C, some browsers will, when fetching C, have a referrer of A, and some will have a referrer of B.

  7. Bounce through a third party! by Anonymous Coward · · Score: 5, Informative

    A lot of websites let you bounce to other sites. Here are some demonstations

    Debian link to aol.com
    Yahoo link to aol.com
    Google link to aol.com
    Goatse link (yes, its true, goatse is useful!) to aol.com

    Hopefully, unless AOL wants to block the internet off, people will get around, and we can always set up p2p based redirection system (ala freenet). To get trough.

  8. Re:They block slashdot too. by mabinogi · · Score: 4, Informative

    No, the maintainers of bugzilla blocked links from slashdot.
    It has nothing to do with AOL.

    They did it because bugzilla is an entirely dynamic site, and an important tool being used by the developers.
    The last thing they want, is 50,000 slashdot users hitting it at once and preventing them from working.

    --
    Advanced users are users too!
  9. Update by Anonymous Coward · · Score: 2, Informative

    Images hosted on Netscape and on FTP servers are now being blocked as well.

    This block seems more intermittent, but it makes you wonder, since Netscape is owned by AOHELL...

  10. All hail Proxomitron! by mad_dog3283 · · Score: 4, Informative

    Stupidity like this won't affect me at all. I use the Proxomitron, and I have the referrer field set to \u (which I think is the default setting). \u inserts the current URL into the referrer field. So, for example, if I hit a link on www.slashdot.org/foo.htm to www.aol.com/foo.htm, the Proxomitron will send www.aol.com/foo.htm and not www.slashdot.org/foo.htm to the server. This is especially helpful for sites that return 404's to requests with blank referrers (since the server always thinks the request is coming from its domain when in reality it may not be.)

    --
    Reprise the theme song and roll the credits!
  11. Re:Killing referers kills EVERYTHING by Anonymous Coward · · Score: 5, Informative

    Then you are dickheads, plain and simple. The HTTP 1.1 RFC explicitly states that users should be able to turn off the Referer header. There are plenty of reasons for doing so. Furthermore, you aren't even using the right status code. It's 401 Unauthorized when you want to deny access, 404 means the content is missing (which it clearly isn't).

    There _is_ a fairly safe way of doing what you are after - let through empty strings and strings with spaces in. This lets through legitimate users who either disable the referer header, or have it set to "blocked by Norton" or whatever, whilst still stopping anyone from usefully using your bandwidth (since most of their visitors will still be providing the referer header).

  12. Re:tinyurl? by Anonymous Coward · · Score: 1, Informative

    "Unfortunately, it doesn't work in anything that ISN'T IE."

    Then why is there a big picture showing how to use it with Mozilla on the homepage. I've tested it and it works fine.

    Unless of course your referring to this particular problem with AOL and some referrer issue?

  13. A simpler, browser independent solution by Anonymous Coward · · Score: 5, Informative

    I don't have a problem with <obligatoryDerisiveness> AOhelL </obligatoryDerisiveness> preventing people from leeching images from their site, but there's a simple way to get around their prevention of direct links to their site: redirect using a META tag, which strips the referer header and makes it look like a direct request.

    For example:

    If you want to link from livejournal.com/myPage1.html to members.aol.com/~myOtherPage.html, then make the link go to livejournal.com/myPage2.html ..... in the header of myPage2.html, include this meta tag:

    <meta http-equiv="refresh" content="0; url=http://members.aol.com/~myOtherPage.html">

    It works accross all browsers and appears to AOL as if somebody just typed that URL directly into the address bar of their browser.

    1. Re:A simpler, browser independent solution by koh · · Score: 2, Informative

      Good idea, but it also screws up the Back button on most browsers, and is not recommended AFAIK...

      --
      Karma cannot be described by words alone.
  14. AOL offers "Complete Internet Access" by flyboy974 · · Score: 2, Informative

    While AOL offers a package that gives "complete internet access", they forget to mention that if you post files online (and you are still online, thus, you should have COMPLETE access, which in my mind goes both ways. After all, FTP will connect back to you), they only go half way.

    Come on. False advertising! They give you a proxied connection to the Internet. You don't have a public IP that somebody can call your BF1942 server from. Full access means you have a public IP address, people can go bi-directional.

    Why call it complete when it's nothing more than proxied?!?

    I once had to visit an AOL office up in the San Mateo hills in the Bay Area. They gave us an address, which we followed. It led to one office building that had an AOL logo as part of a common office space. But, we couldn't find the suite. So, we went to the "information desk", and asked the guy. "They are across the street at Suite XXXX". Come on! So we go across the street. Their offices look loked you were joining the Borg. Leather chairs that were 6" off the ground, all Halogen track lighting. Very secretive. Just plain wierd.

  15. Re:Wait a minute by Anonymous Coward · · Score: 1, Informative
    AOL is singling out LiveJournal; not blocking deep-linking. If they wanted to do that, they would return 404 unless the Referrer pointed to an AOL page. It's apples and oranges.

    Incidently, Windows users can use Proxomitron to strip the Referrer header from HTTP requests.

  16. Can control referer with RefSpoof for Mozilla by kjj · · Score: 3, Informative

    Just head over to here and get the extension. There is even a "Ref=URL" checkbox to make your browser always use the current URL as the referer string so unless websites start blocking themselves, no problem. The good news is that it was just updated to be Firebird compatible as well.

  17. Re:Will this be what kills the referer header? by berzerke · · Score: 2, Informative

    ...Mozilla/Firebird/etc (and Opera) can be easily modified to not send [a referrer header]...

    For Mozilla/Firebird/etc, just install the Prefbar addin. It can be customized to include a send referrer button. Personally, I like the kill flash button the best. Makes those dancing flash ads quiet, while still allowing me to use flash.

  18. LiveJournal Wrote a Workaround!! by ibjhb · · Score: 2, Informative

    From: http://www.livejournal.com/community/lj_biz/195987 .html

    Subject: AOL blocking LiveJournal.com, try two
    From: Evan Martin
    To: postmaster@aol.com
    Organization: Danga Interactive
    Date: Tue, 26 Aug 2003 16:23:40 -0700

    Hello,
    I mailed you yesterday about a problem we're having interoperating with AOL member pages. (I have included that text below.)

    Since we haven't heard back from you, we've instituted a temporary workaround. However, this is not a technically correct solution, nor will it work in all cases.

    In technical terms: we rewrite URLs to AOL member pages to use FTP instead of HTTP so there is no Referer header sent.

    Because we haven't heard back from you, I can only hope that this problem was accidentally caused by some automated system process, and eventually you will be able to fix it. If there is anything more we can do to help speed that process, please let us know.

    -- Evan from LiveJournal.com

  19. Re:Sorta related by jesser · · Score: 2, Informative

    Mozilla nightlies include a workaround to this bug in the AOL server. See bug 210229.

    --
    The shareholder is always right.
  20. Re:Killing referers kills EVERYTHING by j7953 · · Score: 4, Informative
    Furthermore, you aren't even using the right status code. It's 401 Unauthorized when you want to deny access, 404 means the content is missing

    Yes, but...

    First, the "correct" status code would be be 403 Forbidden, 401 Unauthorized is used if "the request requires user authentication" and will cause the browser to prompt the user for login information.

    And for status 403, the HTTP standard (RFC 2616) says that "If the server does not wish to make this information [explaining why the request what not fulfilled] available to the client, the status code 404 (Not Found) can be used instead." The normal use for status 404 is if the server cannot find the requested resource, but according to the RFC it is also "commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable."

    So, either status code 403 or 404 are correct when trying to prevent precise ("deep") links from working. I agree that 403 is preferable.

    --
    Sig (appended to the end of comments I post, 54 chars)