AOL Blocks Links from LiveJournal

Evan Martin writes "LiveJournal.com is an open-source weblog site with over a million users, some of whom use AOL. Last week, AOL began blocking all HTTP requests with "www.livejournal.com" Referer headers. This is a common practice by image hosting sites to prevent off-site linking of their images and 'bandwidth theft'. However, in AOL's case, they're blocking everything, not just images, effectively breaking all links to any AOL member's site--but only from LiveJournal. To be clear: nobody on LiveJournal can even make a link to any AOL member site without getting a '404 Not Found' error. We've also heard reports of the same thing happening on AOL properties (Netscape, Compuserve). This concerns us because we have to deal with the support requests: it worked in the past for our users, and it continues to work for other sites, so our users think it's our fault."

Martin continues: "We've tried to contact AOL three different ways, all without success. We've also told our users to contact their tech support. At one point, an AOL staffer pointed out that FTP access still worked (which is probably because FTP has no "Referrer" concept), and so, as an interim fix, we're rewriting all HTTP URLs to use FTP on the AOL properties where that works instead. This means that users can again host their images on the AOL webspace they're paying for, but more importantly, it means they can simply link to their webpage.

We wouldn't be so upset if they were simply blocking images. Bandwidth use is a valid concern, after all, and we even provide step-by-step instructions for people to configure their webservers to prevent image "theft". However, because they're blocking all access, including regular links, this looks like it's either a mistake, or something more insidious (the conspiracy theorists have pointed out that AOL has just launched their own competing weblog product, also based on "journals").

Although CI Host sued AOL recently for being blocked, we really don't want to do that. We still suspect that this was all just a mistake, and hopefully, by making this public, we'll manage to get their attention, since all our previous attempts have failed."

Comments

F12

[Anonymous Coward]

Enable referrer logging

Re:F12

[Nasarius]

Gotta love Opera :)

Use link referers

[EDA Wizard]

Until this mess gets sorted out, people should use a free link relocator service. Make A Shorter Link would work well for links to AOL pages.

It wouldn't help people with embedded links to images at AOL, but at least it could get people to AOL without any additional clicking.

Solution:

[WCMI92]

Don't use an ISP that is "broken". AOL has little to recommend it.

I use Adelphia PowerLink at home. On the road, I have a dial up account with a local ISP with dial up numbers in the cities I frequently have to visit.

Quick fix for HREFs viewed by MSIE

[inertia187]

That's wrong of AOL, but if you're a LiveJournal user in a bind and really want to fix the links (but nothing else) fast, here's a JavaScript that you can load in all of your pages. You just need to load it once, and the page will work.

Unfortunately, this trick really only works with MSIE. But it's better than nothing.

<script language="javascript" type="text/javascript" src="http://www.martin-studio.com/js_tools/strip_r eferrer.js"></script>

The above should all be on one line. Check for extra white space where the line feed got placed by Slashdot's bug (thanks alot).

It should be strip_referrer.js with no space. Why does Slashdot do that??

Re:tinyurl?

[Jouster]

TinyURL uses a Location: header, which should kill off the referer, yes. But asking everyone to TinyURL their images is a bit much, don't you think? Besides, some browsers don't like having 3XX statuses (stati?) as replies to their image requests, so you'd break some people.

AOL just needs to stop doing that shit. Clamp down on the people transferring 200 gigs in the exhibitionism-community-of-the-week, and leave everyone else alone.

Jouster (My LJ)

Re:tinyurl?

[Chmarr]

It TOTALLY depends on the browser you're using.

If you're on web page A, click on a link to B and it redirects to C, some browsers will, when fetching C, have a referrer of A, and some will have a referrer of B.

Bounce through a third party!

[Anonymous Coward]

A lot of websites let you bounce to other sites. Here are some demonstations

Debian link to aol.com
Yahoo link to aol.com
Google link to aol.com
Goatse link (yes, its true, goatse is useful!) to aol.com

Hopefully, unless AOL wants to block the internet off, people will get around, and we can always set up p2p based redirection system (ala freenet). To get trough.

Re:They block slashdot too.

[mabinogi]

No, the maintainers of bugzilla blocked links from slashdot.
It has nothing to do with AOL.

They did it because bugzilla is an entirely dynamic site, and an important tool being used by the developers.
The last thing they want, is 50,000 slashdot users hitting it at once and preventing them from working.

All hail Proxomitron!

[mad_dog3283]

Stupidity like this won't affect me at all. I use the Proxomitron, and I have the referrer field set to \u (which I think is the default setting). \u inserts the current URL into the referrer field. So, for example, if I hit a link on www.slashdot.org/foo.htm to www.aol.com/foo.htm, the Proxomitron will send www.aol.com/foo.htm and not www.slashdot.org/foo.htm to the server. This is especially helpful for sites that return 404's to requests with blank referrers (since the server always thinks the request is coming from its domain when in reality it may not be.)

Re:Killing referers kills EVERYTHING

[Anonymous Coward]

Then you are dickheads, plain and simple. The HTTP 1.1 RFC explicitly states that users should be able to turn off the Referer header. There are plenty of reasons for doing so. Furthermore, you aren't even using the right status code. It's 401 Unauthorized when you want to deny access, 404 means the content is missing (which it clearly isn't).

There _is_ a fairly safe way of doing what you are after - let through empty strings and strings with spaces in. This lets through legitimate users who either disable the referer header, or have it set to "blocked by Norton" or whatever, whilst still stopping anyone from usefully using your bandwidth (since most of their visitors will still be providing the referer header).

A simpler, browser independent solution

[Anonymous Coward]

I don't have a problem with <obligatoryDerisiveness> AOhelL </obligatoryDerisiveness> preventing people from leeching images from their site, but there's a simple way to get around their prevention of direct links to their site: redirect using a META tag, which strips the referer header and makes it look like a direct request.

For example:

If you want to link from livejournal.com/myPage1.html to members.aol.com/~myOtherPage.html, then make the link go to livejournal.com/myPage2.html ..... in the header of myPage2.html, include this meta tag:

<meta http-equiv="refresh" content="0; url=http://members.aol.com/~myOtherPage.html">

It works accross all browsers and appears to AOL as if somebody just typed that URL directly into the address bar of their browser.

Can control referer with RefSpoof for Mozilla

[kjj]

Just head over to here and get the extension. There is even a "Ref=URL" checkbox to make your browser always use the current URL as the referer string so unless websites start blocking themselves, no problem. The good news is that it was just updated to be Firebird compatible as well.

Re:Killing referers kills EVERYTHING

[j7953]

Furthermore, you aren't even using the right status code. It's 401 Unauthorized when you want to deny access, 404 means the content is missing

Yes, but...

First, the "correct" status code would be be 403 Forbidden, 401 Unauthorized is used if "the request requires user authentication" and will cause the browser to prompt the user for login information.

And for status 403, the HTTP standard (RFC 2616) says that "If the server does not wish to make this information [explaining why the request what not fulfilled] available to the client, the status code 404 (Not Found) can be used instead." The normal use for status 404 is if the server cannot find the requested resource, but according to the RFC it is also "commonly used when the server does not wish to reveal exactly why the request has been refused, or when no other response is applicable."

So, either status code 403 or 404 are correct when trying to prevent precise ("deep") links from working. I agree that 403 is preferable.