Gmail Spam Filter Testing

An anonymous reader writes "What can you do with 1000MB of e-mail space on your Gmail account? One guy, by the name of Aaron Pratt ( prattboy@gmail.com ), has decided to test the spam filters of Google's Gmail service by having his Gmail account blasted with every kind of spam imaginable. He is testing to see how well Gmail's spam filters can sort out the spam from legitamate email (yes, he does get personal emails from people). As of May 25th, he was at about 30% of his Gmail account's 1GB capacity. You can track his progress on his website, http://gmail.prattboy.net (Google cache of this site: cache: gmail.prattboy.net). Here is also an article talking about Aaron's efforts from webpronews.com"

Comments

first spam?

[miketang16]

psh.. i've done this to my friends before.. they didn't need to make a website to ask for it...

Re:first spam?

[Anonymous Coward]

Oh, I didn't know that was you who passed my address along so I could b uy che.ap v1agra! Thanks! Those pi1ls made my p.e..ni.s gr0w 3-5 lnches! It was really very thoughtful of you, Mike.

The Filter is great!

[umrgregg]

Apparently, Google's spam filter even filters messages that aren't there. From the website:

3778 messages were received, totaling 213 MB.
3917 were spam, and Gmail correctly identified 41.9% of these messages.

Fantastic

Re:The Filter is great!

[Anonymous Coward]

No, thats just a classic threaded code bug:

They just forgot the mutex surrounding the two snprintfs... so this user probably got 139 messages in the time it takes to execute snprintf, all spam.

Which is.... about right.

Re:The Filter is great!

[aismail3]

When I add up the figures from May 13 to 19, I get that 4869 messages were received. 4717 of those were spam, and 1820 were marked, so Gmail's success rate was 38.6%.

One of the best things Google/GMail could do

[Anonymous Coward]

Is use the GMail data to operate a checksum blacklist. Obviously, if thousands (or millions) of their users are getting the exact same email, it's probably spam.

Re:One of the best things Google/GMail could do

[kryptkpr]

Spammers have thought of this already, and they send nearly-identical messages.. Ever notice the random strings of letters and/or numbers at the bottom/in the subjects of spams?

Re:One of the best things Google/GMail could do

[lockefire]

Actually, I get a whole lot of emails with the random words and nothing else. I haven't quite caught on to the advertising strategy in that.

Re:One of the best things Google/GMail could do

[Cruciform]

I've been getting them as well.
The only reason I could think of someone sending those around is to bog up Bayesian filters with random crap, possibly lowering their effectiveness.

Any spammmers/spam-experts feel like enlightening us? :)

Re:One of the best things Google/GMail could do

[Anonymous Coward]

Anti-Spammers have thought of this, too. Things like the Distributed Checksum Clearinghouses have "fuzzy" matching.

Google also has enough computer power to generate some sort of Bayesian filter to catch the most common spam system wide, and even a personalized filter on each account to catch the rest.

Re:One of the best things Google/GMail could do

[wo1verin3]

It's a good thing you're not using Outlook. :)

I get those in Eudora and they don't seem to do much, my friends with Outlook however... not so lucky. :)

Re:One of the best things Google/GMail could do

[Xzzy]

My server was set up to forward anything sent to one of my domains to get dumped into a common inbox. I noticed a ways back (before I changed my config to just bounce all this crap) that I'd get a lot of those dictionary emails to random email accounts.

So either it's some kind of probe to find working addresses, or a filter clogger. Or maybe both.

For a few of the random emails I would later start getting "real" spam. Not a majority though.

Re:One of the best things Google/GMail could do

[xandroid]

Try looking at the source -- when this happens to me, I see that the random words are plaintext, and the intended advertisement is in HTML (which I've blocked).

Re:One of the best things Google/GMail could do

[Halo1]

Most of the time, these messages contain both a text/plain section with only random words, and then a text/html part with the real payload. If you use mutt or so, you most likely only see the text/plain stuff. Another trick is using just a text/html section with random text, but also with an image that contains the real payload.

Re:One of the best things Google/GMail could do

[jefe7777]

>> You think they bother?

heh heh...abdolutely.

100 known good addresses are worth 10,000 "who the fuck knows" addressess.

>>It's cheaper to just send mail to everyone

no it's not.

let's pretend you are a spammer, and you want to send out spam.

If you target 1 billion questionable addresses, each time a client has a new campaign, then that's 1 billion pieces you have to deliver. every time.

what if you have 1000 clients? that's 1000 billion deliveries.

do you see where this is going? if you don't KNOW WHAT A VALID EMAIL ADDRESS IS, YOU HAVE TO GUESS.

but what if the first time you send out just a "test" to those billion addresses, and then subtract the one's that bounce.

You are left with 50,000 known good addresses.

that's gold. You now have 1/20th of the load,and you are now serving your clients quicker, a helluva lot less load. you are only using an open relay for 1/20th of the time.

overall a smaller footprint by 1/20th.

you tell me. does it make sense to blindly blast out email?

Re:One of the best things Google/GMail could do

[ryen]

those emails could possibly also contain embedded image tags (known as web beacons). when you open an email and attempt to 'download' the image, some server on the net knows it was you who retreieved the image and has just verified that your email address is active and spammable.

Re:One of the best things Google/GMail could do

[ckd]

They have a much higher ratio of PhDs than Microsoft, or just about anyone short of a hospital.

Remind me not to go to your hospital. I want MDs treating me, not people who can give me a dissertation on ancient Sumeria or something. (MDs who also know about ancient Sumeria excepted.)

Re:One of the best things Google/GMail could do

[dragonman97]

Indeed - while I was doing a lot of spam fighting at work, I reviewed a honeypot I'd set up, and was amazed. I used mutt to review the messages, and found a couple of messages where the text part was a page or two from "The Wizard of Oz" and the nasty offer for some kind of auto insurance or other crap was in the HTML section, replete with hidden hash busters behind color backgrounds. These guys are sharp - they must be paying some smart programmers a lot of money, and it's only sad that they've sunk to such levels.

Re:One of the best things Google/GMail could do

[letxa2000]

Spammer is trying to do two things: 1. break any Bayesian filter used on that mail server/inbox. Adding noise to the filter will allow more mail through as "questionable". This might still be tagged as spam, but not as readily as it would be without the added noise

Except that won't work, as anyone that understands Bayesian filtering will tell you. In the case of every message with "random words" I've checked recently, the random words actually increased the spam score of that message. Why? Because it seems the random words aren't so random and either the same spammer is using the same "random words" over and over or various spammers are using sets of the same words. Over time most of the "random words" they use actually become great indicators of spam since my real email doesn't typically contain the random words they use.

In one recent analysis, 10 random words were inserted by the spammer. He got lucky and 1 of those words actually had a very low score for my Bayesian corpus. Unfortunately (for him), the other 9 words had scores of 99.99%! His use of random words literally nuked any possibility of him getting through my filter.

Anyway, random words will not help spammers get through Bayesian filters. But it seems that many people (both spammers and non-spammers) think it will. But, hey, that's good for me: as long as "random words" is seen by spammers as a viable solution to Bayesian filters, my Bayesian filter will continue to work and will not have to deal with any innovative way to get around the filter (if any exists).

Re:One of the best things Google/GMail could do

[FooAtWFU]

>>It's cheaper to just send mail to everyone
>no it's not.
It doesn't matter how cheap it is when 80% of spam supposedly comes from infected zombie computers. (I'm too lazy to actually LINK to the recent story on this.)

Re:One of the best things Google/GMail could do

[wickidpisa]

It may not increase false negatives, but it has decent chances of increasing false positives which is a much greater problem. My best guess is that spammers are hoping that once enough random words are classified as spam words, real emails with those words will start being classified as spam. If they can force enough false positives, people will start turning off bayesian filtering.

He gave out his e-mail address...

[Anonymous Coward]

... to the entire Slashdot community! Now he's going to be flooded with all sorts of spam and shit. LOL!

Oh... right. :)

Re:He gave out his e-mail address...

[umrgregg]

Notice the reader who submitted the story was anonymous... Gotta love friends who sign you up for spam.

Re:He gave out his e-mail address...

[Algan]

It's not that bad as you think. I posted an dedicated email address to slashdot two times already, just to see what volume of spam I get. Surprisingly, it's only 2-3 messages every other day or so.

Well, I guess I need a booster shot, so here it is: slashdot@hates.ms. Spam away...

Re:He gave out his e-mail address...

[istewart]

I once posted my AIM screenname (IStewart12) to Slashdot and got a total of one message from a concerned individual warning me about the flood of IMs I was now likely to receive. Must not have been a very active thread.

whining?

[Gothmolly]

What's Google going to do to protect its users from mail bombs?

Now you're complaining that your free, 1GB-limit, access-from-anywhere email service could be mailbombed? Live with it. If Google "decides" anything more about our emails, we put on our tinfoil hats and scream. If we broadcast a bogus email address, obtained from gmail for clearly sinister purposes, and it gets mailbombed, we whine that Google doesn't "protect" us. Whats the story, or are we all just schizophrenic?

Don't want that "vulnerability"? Don't use Gmail!

Re:whining?

[supersnail]

I don't think its about protection just practicality. Google offers a SPAM filter the littel pratt tested it and found it wanting.

I think its more of a problem for Google than the end users. The whole Gmail "get a gigiabyte of memeory free" business model is predicated on most people using only a small fraction of that Gigibayte but felling good about the capacity being there. If I open up a gmail account, get p*ss*d of with the spam and go elsewhere without closing the account the 1G will fill up with spam in a couple of months, Google will end up storing terabytes of spam for cutomers who no longer use the service.

Re:whining?

[Pharmboy]

Now you're complaining...

That is his JOB, to point out shortcomings of the system. He is a tester, and he is doing it for FREE. Google doesn't want testers who get 3 emails a day, they want people to test the living shit out of the service and point out what is wrong with it. Everyone knows Google will try to fix all the bugs, so all the press, good or bad, is still good press.

If Google barfs when handling 999 messages in 4 minutes during testing, image when several million people have gmail accounts. Fortunately, now Google has an even to look at to see what the problem is. When you are trying to harden a system, YOU MUST BREAK IT OVER AND OVER AGAIN, to see where it is weak. This is what is happening.

My impression is that the tech's at Google are spending a significant amount of time saying "oh shit, never thought of that, cool." which is the ENTIRE REASON FOR TESTING. They can't think of every situation by themselves. This is also the entire concept behind "open software is more secure". Google's gmail is going to have bugs at this stage and lots of them, period. Google knows this, hell, everyone knows this (this is why its in testing, and not open to the public yet, duh)

It not whinning, its stating the facts, which Goggle obviously WANTS him to gather, as a TESTER. Seems to me that he is going beyond the call of duty to test their servers, since he is spending a fair amount of his own time.

Re:whining?

[ovlaski]

So what? If they have terabytes and terabytes of spam, they have a huge database to teach their filters with.

Re:whining?

[thogard]

If I offer 10 of my most leeaching customers 1 gig of space, I will need 10 gig of space... or will I? How much of that will be duplicated between at least two users and how much of it will be used by all 10? Remember Google already has copies off allmost all the useful stuff on the net. If you grab some random web page and mime attach it to email, thats going to waste space in my mailbox but if google can figure out that they already have all the images, as well as the text, its going to compress down to very little. For the 1st customers it requires a massive increase in needed disk space but at some point it starts dropping off. Sort of like how much stuff they have to index for the web and image searches.

Re:whining?

[cmacb]

Actually the TOS for Gmail says that doing things to attract spam is a violation, so they could just close the account on that basis. Also, if you don't sign on for a certain period of time (a few months I think) the account gets deleted. I had a Yahoo ID for years before I ever knew there was an e-mail address associated with it. I never read the mail associated with my AIM id and I probably still have free hotmail and a few other things like that floating around. Failure of these companies to delete idle accounts is what causes all the good names to be taken. I think Google is more on-top of this than many of the others.

Re:whining?

[Valluvan]

Not many are as gregarious as Pratt. I've been using gmail for some time now. I must say google has done a pretty good job with their spam filters. For not-high-volume users (which most people are), gmail works much better than other email providers (i have yahoo, ureach and hotmail accounts which I use regularly).

Of course, google should improve and filter out the occasional crap I get too. And also offer 1 TB.

Re:whining?

[AKnightCowboy]

When you are trying to harden a system, YOU MUST BREAK IT OVER AND OVER AGAIN, to see where it is weak.

Slashdot operated under that philosophy for the first 2-3 years of it's existence. ;-)

Re:whining?

[Beryllium Sphere(tm)]

>The whole Gmail "get a gigiabyte of memeory free" business model is predicated on most people using only a small fraction of that Gigibayte

Why?

Google uses commodity IDE drives. Those retail for about fifty cents a gigabyte. Google's not paying retail.

I read a quote from a Googleperson that by the time the drive is installed in a system, powered, cooled, backed up and administered Google is paying two dollars for a gigabyte.

Good point about the problem of abandoned accounts, which won't bring Google any ad revenue. Wouldn't be surprised if they start euthanizing inactive accounts.

Re:whining?

[StrongAxe]

Good point about the problem of abandoned accounts, which won't bring Google any ad revenue. Wouldn't be surprised if they start euthanizing inactive accounts.

Both Yahoo and Hotmail automatically close and erase free mail accounts that are inactive for 30 days. I wouldn't be surprised if most other free email services had similar policies.

If this guy has used 30% of his capacity...

[Dagny Taggert]

...how many e-mails has he recieved in total? I've kept spam for six months before and it totaled less than 100MB...and I get a cubic buttload of crap daily.

Re:If this guy has used 30% of his capacity...

[Zeebs]

and I get a cubic buttload of crap daily

God damned metric system.

gmail still beta

[ryen]

isn't gmail still in 'beta' stages? if so, isn't a review of spam filtering techniques a little premature?

Re:gmail still beta

[waddgodd]

>isn't gmail still in 'beta' stages? if so, isn't a review of
>spam filtering techniques a little premature?

What part of Beta TEST escapes you here?

Not a fair test

[SWroclawski]

He's not counting all the mail that Google is rejecting and not even being allowed in for further classification.

Re:Not a fair test

[Plutor]

Is there any evidence that Google actually does this? I would think that would be terribly non-transparent. Auto-deleting email that it's "really sure" is spam is still dangerous. Even the best-trained Bayesian filters will have false positives sometimes. Is this just random theorizing, or does GMail really fail to deliver some emails it thinks is spam?

Re:Not a fair test

[SWroclawski]

Any evidence that they reject mail for various reasons? I'm sure there is. You can go ahead and see which RFCs they're in compliance with and which they aren't.

If you don't have a PTR record associated with your host, try to send mail to them, or malform your EHLO or something else.

You don't need to be "really sure" mail is spam- I'm talking about doing things like standards complaince checking, which will result in mail being rejected at delivery time.

Is this just random theorizing, or does GMail really fail to deliver some emails it thinks is spam?

There's no reason to get insulting. RFC 2821 has a number of requirements for delivery of mail that many services ignore.

Should be interesting, what filters?

[Clinoti]

Can anyone provide a link or source to the kind of filters google has working on gmail?

I'll help

[L. VeGas]

Let's all send him an email and ask him how it's working out.

News...

[somethinghollow]

"Here is also an article talking about Aaron's efforts from webpronews.com""

Since we are talking about spam and obtaining more spam, I don't know if I should read the site the article is on as "web pro news dot com" or "web pron ews dot com"...

I guess I'll figure it out sometime.

Not that impressive

[chrisgeleven]

Seems like Gmail only filters approx. 50% of spam. That is not very impressive, since the top anti-spam software and e-mail clients (such as Outlook 2003 and Mozilla Thunderbird) can easily reach 95% accuracy in spam filtering.

I am starting to second guess whether I should transfer everything to my Gmail account.

Re:Not that impressive

[Apiakun]

Don't forget that this is google's first foray into mail software, and it is still in beta. I have so far gotten very little spam in my gmail inbox.

Re:Not that impressive

[XO]

Sure, but those will also mark virtually every legitimate email as spam, as WELL. Yeah, you can have 95% accuracy... but then you have to go through your hundreds of messages marked spam just to find your real email!

(example, after two weeks of using spam-assassin, it decided that every e-mail sent to me was spam.. i no longer received anything in my Inbox, everything was transferred to the Spambox. It took me another two weeks tweaking spam-assassin's kill rate down to about a 50% accuracy, and now i actually receive all my emails.)

Re:Not that impressive

[peeping_Thomist]

I have so far gotten very little spam in my gmail inbox.

What was that address again?

Re:Not that impressive

[Kredal]

tikora@gmail.com, I think.

Mine is kredal@gmail.com, if you're interested. (:

Re:Not that impressive

[ravydavygravy]

Sure, but those will also mark virtually every legitimate email as spam, as WELL. Yeah, you can have 95% accuracy... but then you have to go through your hundreds of messages marked spam just to find your real email!

Rubbish - I've used thunderbird for many months now, with an account that gets quite a bit of spam. I have yet to see thunderbird make a wrong guess at whats spam and whats not. If anything, thunderbird is more likely to go the other way - allowing spam through - than deleting real email.

Re:Not that impressive

[furball]

Mine's gdnguyen@gmail.com.

Please only email me if you're barely legal and running a webcam. Thank you.

Should gmail be filtering all emails

[Anonymous Coward]

If I understand what he was talking about on his site, what he cansiders as spam partialy legitimate mailing lists and are not realy spam even if he did not personally sign up for them. IE (Me signing him up for the gay porn of the month club.) He may not want it but, I signed him up.

I just want....

[AviLazar]

to be able to reserve a name without numbers attached to it.... Damn it's going to be a race :(

What is the big deal?

[Zugot]

Mozilla Thunderbird or Spamassassin will filter at least as well or even better. Is this just a test to see how quickly we can fill up gmail's disk?

Is this the AventureMail guy?

[magefile]

The guy who got booted off AventureMail (2GB free) for trying to test their spam filters? The story is on Kuro5hin, if anyone wants to see it.

My own gmail testing

[Twid]

I did some testing of my own. I forwarded a ton of spam from my personal account to my gmail account, just to see what would get through and what would be filtered. For me, gmail was really effective, but strangely, one Nigerian e-mail scam mail didn't get tagged.

It was from " Mr Jubril Udeh Manager of Credit and Accounts Department of North Atlantic Securities Sarls Lome-Togo Republic."

Now, the funny part is not that the mail made it through, but that google also decided to show me contextual ad's on that account. Currently, the ads are:
- Payroll Cards a Poor Substitute for Checking Account
- Tips for Tackling Check Fraud
- Sophos hoax description: Ethiopian airline letter
- FAP non-US Investment FAQs

In the past the mail has also shown me ads on how to open an off-shore bank account. I'm glad google is willing to help me with the $10.5 million dollars that I'm about to receive! :)

Spam is always personalized

[Sulka]

Checksums are nearly useless against spam. It only takes one byte to change the checksum value and probably more than 90% of spam contain a personalization code to check which addresses are functional. Different code = different checksum.

This doesn't mean it wouldn't be possible to create a system which would automatically detect individual spam messages based on tagging known spam, you just have to be smarter about the detection than just plain MD5ing the email body.

Re:Spam is always personalized

[GlassUser]

gzip it and compare the files. a short tracking code will make a negligible difference.

Re:Spam is always personalized

[Thuktun]

gzip it and compare the files. a short tracking code will make a negligible difference.

Not necessarily.

Lempel-Ziv based algorithms, like the one used by gzip, build a compression dictionary on the fly. Any "personalization" added to the message will affect the dictionary to varying degrees from then onward. If it's near the beginning, the personalization would greatly skew the selected dictionary identifiers. Though probably this would have little effect on the actual compression of the data, it would radically change the representation of the compressed image. The farther this personalization is from the start of the data to be compressed, the less effect it will have.

Wow! Lot o' spam.

[justkarl]

As of May 25th, he was at about 30% of his Gmail account's 1GB capacity

30% * 1000MB= 300MB of spam? I don't think I've got half of that in my life. Maybe 100MB of spam lifetime.
SO, let's do more math. Avg. spam message=5kb. Therefore, 5kb/300MB(by 300*1024)=61440 messages? Am I right? That is a whole bunch!

About spam and blocking

[AviLazar]

While we cannot block every domain name (i.e. if you get spam from $#(*$#sexphreak@yahoo.com) because it will alienate your legitimate contacts, there are many domain names that we can block (i.e. @spam-your-gmail.com). Yahoo provides email/domain name blocking, but limits this to 100 (unless you are paying). Do we know if gmail will have this limitation?
-A
*just for those who didn't know, the above domain names and email accounts are random, any resemblence to an actual domain or email account is purely coincidental, and if you choose to do so, you should sue /., not me :)

Re:About spam and blocking

[OiPolloi]

Gmail allows you to create any number (at least they don't seem to have any limit) of "filters". These are rules that allow you to manage your messages based on sender, recipient, subject, if the message has an attachment, if it has certain words, etc.

So this allows you to block some domains, if you'd like.

Re:About spam and blocking

[stevesliva]

I've found whitelists, combined with treating everything as junk, to be far more useful than blacklists.

1gb Relieves Spam Concerns

[osewa77]

I have subjected my e-mail address, afriguru@gmail.com to the same abuse. by redirecting all e-mail addresses that recieve lots of junk mail to this one and posting the address unprotected to lots of websites and newsgroups. At the initial stage, a lot of 419 scam mails got through, but now I hardly get any spam. No false positives for me so far.
_____________________
Seun Osewa, Abeokuta Nigeria

Hmmm.. weird stats...

[Mz6]

Hmm.. well let's see...

His last week stats are:

3778 messages were received, totaling 213 MB. 3917 were spam, and Gmail correctly identified 41.9% of these messages.

Something is off... Unless his spams contain attachments, this says that each of his emails were 17 MB in size each.

I mean 17.73708.. This is /. afterall. :)

Re:Hmmm.. weird stats...

[Satai]

no, you inversed it. You want MB/message, not message/MB.

3778 messages / 213 MB = 17.37 messages / MB
213 MB / 3778 messages = 0.0564 MB / message

So that's pretty reasonable.

Re:pre-emptive strike theory

[umrgregg]

Right! My only idea is that Google's technology is so advanced, it filters messages before they are even sent. It's gotta be a result of faster-than-light calculations. Boy, I'm gonna buy me some stock.

gmail spelling

[Anonymous Coward]

>legitamate

How about having Slashdot editors/Hemos test the gmail spell checker too?

This won't work for me...

[Chuck Bucket]

This won't work for me, how will I get emails like:

• Home loans and refinancing
• Proven techniques help you find a date tonight - guaranteed!
• suuuper streeeeeeetch your coock
• Drive that new car today
• Give the girl what she needs
• STRAIGHT TALK ON HAIR TRANSPLANTS
• SEXUALLY-EXPLICIT: Rise N Shine, there all here
• Your Degree by Fedex shipped
• Make your man hood work right
• Rooooock Haaaaard Ereeeectiooons In 60 Seeeeeecooooooonds
• Sexually Explicit: At home mom's nude on cams
• Free Phone Free Shipping Easy Qualify
• get the p .e. nis si. ze she wants
• Clearance on 6 MegaPixel DigiCamera
• FWD: Ciialiss quazy DISCOUNTS - this is better then viagra, $2.0

** NOTICE: all subjects taken from my Yahoo email acct, from emails recieved this weekend. now you see why I run my own mailserver at home? **

CB

Re:This won't work for me...

[Chuck Bucket]

When signing up for my DSL I made sure all servers were OK to run. Once I had that I setup my mailserver, learned how to admin it, learned how to run Spamassasin, gave accounts to friends/family and now I have about 10 users that hit it everyday.

So yeah, make sure it's OK with yr ISP before signing up, and then you're free to do what you'd like.

CBV

How is he compiling stats?

[HellKnite]

Anyone know how he's pulling the numbers off the page? Is there some kind of sneaky back-end that we can get stats about our account with? Is he manually entering all this info? Or maybe some kind of "screen-scraping" techniques to pull the data off the page... hmm...

I guess because his stats are about 2-3 weeks behind, it would indicate that things are leaning towards the manual procedure...

0% Spam

[yuri]

Spam is unsolicited, so google should filter none of his mail.

This guy solicited it.

Filtering could use some help

[csimpkins]

This guy gets thousands of Spam mails without a problem, yet I can't receive a simple HTML attachment without the mail being rejected (552 Illegal Attachment). Hrmm...

Lack of updates?

[Xiadix]

Did anybody else notice that his site hasn't been updated in almost a month (May 25)? Seems his project is no longer working. I wonder if Google booted him.

KevG

It's going to get a lot better...

[waytoomuchcoffee]

For those of you that don't have Gmail yet, there is a little "Report Spam" button you can use to, well, report spam. When Gmail gets a few million users, and even 1% use this little button, you are going to see the spam detect rate skyrocket.

Cache?

[Freon115]

Do you really expect the Google servers to go down because of /.? ;)

Re:Cache?

[leo_llew]

Obviously not, they provided a link to the GOOGLE Cache ;)

Re:Cache?

[Calamity Jane]

The cache link is pointing to the cache of his website, not of Google's.

Viola

[doodlelogic]

If I could stop all the spam I get...I'd feel like a whole string quartet!

Re:How to never get spam

[mumblestheclown]

Hi! And welcome to the Internet! We're glad to have you aboard.

Just to get you started, I'll give you a quick hint: virtually every internet discussion on spam includes some high and mighty moron that claims that by not giving out his email address, he never gets spam.

The problem is, that for every one of those, there are plenty more who follow the same precautions and yet get plenty of spam to those accounts for a variety of reasons. Clearly, your soution is not the answer to "how to never get spam."

A good rule for using the internet is to read a few discussions before you post. This way, you will be less likely to post something that makes you look naive. So sit back, relax, and enjoy a steaming hot cup of STFU while you read and learn!

didn't somebody already sort of attempt this?

[cks3]

Oh, wait, it was me! http://slashdot.org/comments.pl?sid=105335&cid=896 5252

Eh, I only got 180MB worth of email and spam out of the deal though, before I decided to delete the account. The Gmail Spam filter was rather horrible at the time; catching only the most tried and true SPAM, letting tons of other SPAM through, and then randomly flagging legitimate messages from people whom it had not flagged before. I think it has improved some since then.

Wow

[EaterOfDog]

His wang is going to be huge!

Paid yahoo is better

[Avumede]

I pay the $20 for extra Yahoo email, and I have to say that their spam filtering is much better than gmail's right now. I have about 10 spams a day to clear out of gmail, where with Yahoo it's more like 1, often 0.

People that don't pay for Yahoo don't seem to get such good spam filtering, though.

Google can definitely do better.

I redirected an old address

[FooAtWFU]

I redirected an old manager@(two letters here).net site so Gmail gets a carbon copy of all the spam sent there (it's lots, trust me). At first it seemed that my Thunderbird Bayseian filters were doing better, but the trend seems to have reversed lately.

No, I'm not keeping proper statistics. =b

Calculations?

[haxor.dk]

So, in less than a month, he has recieved in excess of 300 Megabytes of useless junk ?

I think somebody needs to recalculate axactly how much bandwidth go to waste because of this SPAM plague. The cost in global comms traffic must be staggering!

More focus on false positives.

[ron_ivi]

Reviews of spam filters always seem to focus on how much stuff they block.

The consequenses of blocking a non-spam email are so much worse (parent not hearing from kid. the customer that would have saved your startup.) than a spam getting in, I wish the spam filter reviews would focus on those.

Re:More focus on false positives.

[Anonymous Coward]

false positive : spam getting past the filter ratio...

A false positive is not one of spam getting past the filter, it's one of non-spam getting blocked.

I.e. the filter says it's spam, and it isn't - in the same way that a false-positive medical test says you have a virus even when you don't.

Re:More focus on false positives.

[einTier]

False positive = condition you are testing for comes up positive, when it should be negative.

False negative = condition you are testing for comes up negative, when it should be positive.

Put in the context of a spam filter, it depends on whether you are testing for spam or for legitimate emails. If you are testing for spam (if spam then...), a false positive would be an email that is not spam getting sent to the spam folder or deleted. A false negative would be spam that lands in your inbox.

Now my friends are spamming me

[mparaz]

"Please invite me to GMail!"

Dumb question about SPAM filters..

[StressGuy]

I have Mozilla, it has a Bayes SPAM filter. Lately, it's been getting fooled more and more. The messages that make it through have one or more of the following features:

1) Several intentionally mis-spelled words

2) Lots of text in white (so it's invisible or nearly invisible)

3) Message in .GIF form only - no plain text.

Could you add filters that look for, say, more than 10% of the words mis-spelled, text font nearly equal to background color, or no actual text in message? These would take effect in addition to the existing Bayes filter.

Re:How to never get spam

[Frobisher]

Which is why my email address is llllllllooong. 20 characters before the @ sign. I don't post it anywhere and I think 20 chars is outside the range of such brute force methods. I've been spam free for about 2 years. And I mean spam FREE. I get NOTHING to my junk mail folder. Its marvellous!

Aventuremail not as tolerant

[dirvish]

I tried to do the same thing with my AventureMail account but AventureMail wasn't cool with it. They deleted my account! You can check out what little data I collected before the account suspension and read the emails to and from AventureMail about the merits of the account suspension at http://3fingersalute.net/aventuremail

Yale Story

[dirvish]

Here is a discussion from Yale's LawMeme on the legal ramifications of Prattboy's experiment. Does asking others to sign you up for spam count as an opt-in?

New spin on the "word salad" strategy

[Scott Richter]

Except that won't work, as anyone that understands Bayesian filtering will tell you. In the case of every message with "random words" I've checked recently, the random words actually increased the spam score of that message. Why? Because it seems the random words aren't so random and either the same spammer is using the same "random words" over and over or various spammers are using sets of the same words. Over time most of the "random words" they use actually become great indicators of spam since my real email doesn't typically contain the random words they use.

Right, and my Thunderbird Bayesian filter catches all of those word salad approaches. But they've come up with a new one - what I call the "encyclopedia attack."

What they do is copy an encyclopedia entry and put it at the bottom of their spam. The thing is usually a few paragraphs long, so that textually it dominates the message. The subjects are fairly random, and are occasionally educational ;)

The problem is that the text of this doesn't trip the "too many strange words" flag that's used for word salads. My Thunderbird filter is really having trouble with these. Anyone else having trouble with these spams?

Re:New spin on the "word salad" strategy

[mibus]

Anyone else having trouble with these spams?

Surely it's the people who aren't having this problem that you want to hear from - they're the ones with good spam filtering ;-)

Improvement over time.

[Jett]

I've had a gmail account for almost 3 months now. In the first month I got 3 spam messages, they all made it thru the filter. Since then I've gotten 5 more, only 1 of which made it thru. It's not statistically significant yet, but to me it feels like the filter has improved. I'm already up to 5% of my 1gig too...

Image-Based Spam and Checksums

[BarefootClown]

What about vetting at least the image-based spam for checksumming? Scan the e-mail for image links (or images included inline). If there's a link, check it against the known list of spam links. If it's in the list, mark the message as spam. Spammers will quickly figure that trick out, though, so step two would be for Google to follow those links, and retrieve the images. Run a checksum of the image file itself; if there are a lot (say, a thousand) messages including the same image, tag it as spam. This combines spam filtering with the fun of reminding spammers that Google has an order of magnitude more bandwidth than they do. Use their own messages against them: the more you spam, the bigger the Slashdotting (Googling? Alas, that word's already taken.)

For bonus points, keep the downloaded images in the Google cache; keeps them available for the mail user, alleviating the load on the sending site for legitimate messages, and keeps them available for, well, the Google cache.

The Solution to Spam

[vakuona]

I have 3 ideas that may overcome spam.

This may require an overhaul of the email system though. One may be to have multiple addresses bound together. So you would give one email address and only "authenticated" or approved contacts could get your second address. Now sending an email simultaneously to the two email addresses would result in the email being delivered directly. mail sent to only one of the 2 addresses would be delivered as per normal, and would be subject to the normal filtering. But i guess the spammers would find ways to get both addresses too and defeat that, but it sould be doubly difficult, if not actually an order of magnitude more difficult. How may people get the same spam on different email addresses? This could be useful.

The other is to hit spammers where it hurts, audience. By rolling out a proper ad delivery system (yuck) which was separate from email, if people used their email less for getting information about products, but had it collected by some RSS type system, the spammers would be left with a dwindling audience unless they switched too. The ads would be strictly opt in.

Or mail collection rather than mail delivery. If people collected mail rather than got it delivered to them, they could in theory just not collect spam. why would anyone collect spam?

Lastly is education. if people kept their own whitelists of approved mailers, they could in theory get rid of most spam by keeping good whitelists.