Slashdot Mirror
New Spam Frontier: Referer Logs
geoffsmith writes "Wired News is reporting that spammers are using referer logs as a cheap new way to
spam small sites. Anyone running a website has probably already seen this phenomenon; I'm thinking of writing a script to remove these entries from my access_log by looking for hits that don't grab my images. (sorry lynx users!)"
The entire internet will eventually go down in a deluge of spam unless it is made illegal and the laws are enforced!
"I'll adapt or I'll discontinue. I'm not planning on becoming the major annoyance of the blogging world.... I'm not too worried my reputation. Marketing is all about being innovative, different, adaptive, taking risks and knowing how to use the technology. I'm trying to be all that."
Heh, it's funny that this guy can make this statement and expect to be taken seriously. It's even more pathetic that he actually thinks he's "innnovative".
I don't know if i'm the only one, but has anyone else who doesn't filter their e-mail noticed a drop off in the amount of spam they recieve? For about the past 2 weeks, the amount of spam in my hotmail inbox has dropped from about 40 to around 15 a day. Anyone else had something similar to this happen?
"Sic Semper Tyrannosaurus Rex."
(sorry lynx users)
Don't worry. It's highly unlikely that any of the 4 current users will visit your website anyway.
Windows users are complaining that Microsoft is filling up their computer's System Event Log with spam about illegal exceptions and page faults.
I don't know who started it - but I find it very odd that browsers send referer info by default. Why? It does not provide anything extra for the user but problems. It is not once or twice that you find URLs to "confidential" pages if you browse through your webserver logs. And... I bet 95% of web surfers do not even know that they are sending this information all the time. Is there really any reason why the default is to send the referer info? I have seen people riot on much less important privacy issues. Why not about this? The referer plague exists in almost all browsers - and only in few browsers you actually can easily turn it off. What's going on?
255.255.255.255 - - [27/Oct/2002:00:00:00 -0000] "GET /perfectly/valid/page/at/yoursite.html" 200 2467 "http://www.wilddonkeysex.com_for_Wild_Donkey_Sex/ " "(SpamBot5000)"
and then people looking at the report would say, "hey, the page at wilddonkeysex links to my perfectly/valid/page and it's getting like 500 hits a day from there, woo! let's click on that url and see what the link to my page looks like!"
-calyxa
Decay! Decay! Decay! -Helium
[Wishful thinking mode ON!]
This implies that there are, maybe, all of 10,000 suckers who keep every spammer on the planet in business. If we find them and cut them off, spam response would drop to about 1 per billion and there's just no way they could make any money off of that.
Dyolf Knip
Yes, referrer information makes an excellent authentication scheme for highly confidential system dealing with transfer of mission critical information. ... Just also check for a magic string in the user agent and voila! trusted computing reinvented. To make it unhackable - just add a few more levels of obfuscation. ;))) The sad part of this, is that I have actually seen authentication schemes like this. Don't know whether I should cry or laugh :)
There are many reasons, mostly for those who program websites. Sometimes you don't want people to see a page before another. this could also be solved with cookies, but some blocks those too.
Then there is the statistics, learn how people navigate around your site. referer can help you see a pattern and improve your layout.
Also it can prevent bandwidth hogs, mostly a issue with ad. graphics and pron sites where people use graphics from others servers on html pages on their own sites but also on free servers where people place graphics and files and link to those directly without using any html and then not showing any of the free servers ad's which provides them with money to run the sites in the first place.
my sig
I think it's more than the web site's owner clicking on the page - a lot of bloggers post a list of "top referrers" on their web site as a way of thanking the referrers, and therefore they generate a lot of traffic to their referrers from their own visitors.
It's nice, as a site operator, to know where your guests are coming from. A good portion of my visitors come from Google and other search engines. The referrer log lets me know what they were searching for, and in nearly 95% of the cases they were looking for a specific topic on my site. I can send them directly there, give them a specific welcome message if they haven't been to my site before, etc.
Furthermore I can restrict traffic for some areas of my site (like some sites that block links from slashdot) for particular reasons or uses. "You just came from the page of an associate and are able to receive a discount." "This page is restricted to users of xyz.com. Please go there first."
Lastly, it protects my image content. My images are not stellar, and yet other sites continue to use them on their pages. I can use the referrer to limit the damage done by only allowing the images to be referred by pages from my own site.
Referrer information may be annoying to you, but it's an extrememly useful tool. If taken away one restricts opportunities for the site operator to personalize and protect content on their site. Not a huge loss, but it isn't really as great a privacy issue as you seem to believe.
-Adam
It is extremely useful for security purposes.
No, not the security most people are thinking of. Checking to see if the user came from FeedBack.html before executing FormMail.pl is no security, since spammers can forge any referer they want.
I'm talking about security which stops a human user who is logged in to a particular website from being tricked into performing actions they didn't authorise. For instance: I log into my server's adminsitrative area. Then, in another window, I browse someone's blog. And I click on their "search" button. As it turns out, this search button is a trap, which sends me to my own admin area with a command to delete someone's account. I'm logged in, I have a valid network address, I'm active, there's no problem. Except that fortunately my browser sends "Referer: www.blog.org" instead of "Referer: www.admin.com".
That's why referer info is useful: to prevent a user from being hijacked.
Slashdot monitor for your Mozilla sidebar or Active Desktop.
This is so damned annoying. If I'm searching for some specific information, I don't give a damn about your idiotic welcome page. I don't care what your website is about or what you have to say on your other pages - all I care about is the specific technical information that google told me you have.
More and more, I'm finding myself using googles cache instead of clicking on the actual links. I know you couldn't care less about my insignificant browsing habits, but the more people start doing annoying crap like this, the more people start using google instead of the web.
"This page is restricted to users of xyz.com. Please go there first."
Do you realize how stupid this is? You're trying to control how I use my browser. Of course I'm not going to go to xyz.com and try to use their idiotic navigation looking for a link to you. You're simply advocating another form of advertisement and I'm not interested. I care about the data you're providing, not how you're getting funded.
I can use the referrer to limit the damage done by only allowing the images to be referred by pages from my own site.
And this is, of course, broken behaviour. Did you know that when you open a new link in Netscape/Mozilla that the browser does not send any referer at all? This means that I can't open your images in new windows and I'm constrained to view your images one at a time. Also, the some browsers change the referer for images when you "save" images (eg, right-click and choose "Save as..." may not send the referer you're expecting).
If taken away one restricts opportunities for the site operator to personalize and protect content on their site.
If you're using this to restrict content to your site ... well, forget it. If you have something I really want, I'll open up a terminal and telnet to port 80. Yes, this is indeed effective restriction. (Quiz to see if you really know what you're doing: how would you set it up so that you know that a user has previously visited another site, with cryptographic confidence?)
As for "personalizing" content, please stop. The only times I've seen that word being used in a web context is to personalize advertising (and also restricting content because I'm not using IE, but don't get me started on that). I've never seen anyone "personalize" a site in a useful way, eg, "You're a C programmer who writes Solaris kernel modules, so you're probably not going to spring for my Herbal viagra scheme and I'm going to cut the marketing BS and give you only useful information."
Why do these "blogs" even keep logs of referer links? This is pure narcisism (and more importantly, a waste of disk space - even though disk is cheap, it's still worth more than someone else's paltry feeling of acceptance). If you're going to say something, just say it. Don't base your life around how many people like what you say. "Ohh, somebody linked to my journal, that means I'm special and I can now feel good about myself." Ahh - get a life.
I swear, "webmasters" piss me off.
I actually bought something from a spam. It was a slightly topical T-Shirt that I thought was clever. Cost me $15 (PayPal).
The guy who sold it to me was obviouly a late teen, and was making ok money selling shirts at about $5 profit per when I called him.
I think most geeks have no problem with spam itself (in fact targeted spams that interest me often get clicks, I get about two of those a year), they have a problem with the number of scams that are sent using spam.
I live in a giant bucket.
Backlinking, or posting your referral logs, is doomed to failure and rightly so. It's just a glorified way of making your site into a link farm, with the expectation that your fellow bloggers will do the same. It is serendipitous that this practice is open to 'abuse' although I would never call the abusers spammers. They are just utilizing a method for submitting data that the site owners themselves have provided. I don't see any reason to call this 'spam' since the site owners are inviting users to submit data through HTTP referral headers.
Also, this quote from the article is ludicrous: "bloggers are not thrilled, even though they ruefully admit that the log spamming may falsely boost their ranking on some search engines."
There is no search engine that bases your rank on the number of sites that you LINK to. I believe the bloggers actually mean that they're sorry to see their backlinks (read: link farms) go, since those do in fact raise search rankings. What a travesty- Sites may have to rely on the actual quality of their content, rather than trading links!
Amidst the alarmist cries in the article, "spammers will destroy our practice of posting referral logs," nobody has even mentioned that there is a ridiculously easy technical solution. Before posting a referral link, why not just have your software visit the referring site and detemine if it actually links to your page? This will defeat the referral advertisers.
For now I'll delete the entries by hand, but if this increases it could get really annoying.
AlpineR
And this is, of course, broken behaviour.
So do you have an alternative proposal to prevent resource (i.e. bandwidth) theft? That is a very real problem, and no amount of arguing that the current solution is "broken" will get people to change unless you provide them an alternative.