Slashdot Mirror
Gmail Spam Filter Testing
An anonymous reader writes "What can you do with 1000MB of e-mail space on your Gmail account? One guy, by the name of Aaron Pratt ( prattboy@gmail.com ), has decided to test the spam filters of Google's Gmail service by having his Gmail account blasted with every kind of spam imaginable. He is testing to see how well Gmail's spam filters can sort out the spam from legitamate email (yes, he does get personal emails from people). As of May 25th, he was at about 30% of his Gmail account's 1GB capacity. You can track his progress on his website, http://gmail.prattboy.net (Google cache of this site: cache: gmail.prattboy.net). Here is also an article talking about Aaron's efforts from webpronews.com"
Is use the GMail data to operate a checksum blacklist. Obviously, if thousands (or millions) of their users are getting the exact same email, it's probably spam.
Can anyone provide a link or source to the kind of filters google has working on gmail?
Let's keep in mind that patents are in place to keep lawyers employed and keep them litigating. -CatGrep
The guy who got booted off AventureMail (2GB free) for trying to test their spam filters? The story is on Kuro5hin, if anyone wants to see it.
As of May 25th, he was at about 30% of his Gmail account's 1GB capacity
30% * 1000MB= 300MB of spam? I don't think I've got half of that in my life. Maybe 100MB of spam lifetime.
SO, let's do more math. Avg. spam message=5kb. Therefore, 5kb/300MB(by 300*1024)=61440 messages? Am I right? That is a whole bunch!
While we cannot block every domain name (i.e. if you get spam from $#(*$#sexphreak@yahoo.com) because it will alienate your legitimate contacts, there are many domain names that we can block (i.e. @spam-your-gmail.com). Yahoo provides email/domain name blocking, but limits this to 100 (unless you are paying). Do we know if gmail will have this limitation? /., not me :)
-A
*just for those who didn't know, the above domain names and email accounts are random, any resemblence to an actual domain or email account is purely coincidental, and if you choose to do so, you should sue
I mod down so you can mod up. Your welcome.
I have subjected my e-mail address, afriguru@gmail.com to the same abuse. by redirecting all e-mail addresses that recieve lots of junk mail to this one and posting the address unprotected to lots of websites and newsgroups. At the initial stage, a lot of 419 scam mails got through, but now I hardly get any spam. No false positives for me so far.
_____________________
Seun Osewa, Abeokuta Nigeria
His last week stats are:
Something is off... Unless his spams contain attachments, this says that each of his emails were 17 MB in size each.
I mean 17.73708.. This is /. afterall. :)
Hmmm.
Anyone know how he's pulling the numbers off the page? Is there some kind of sneaky back-end that we can get stats about our account with? Is he manually entering all this info? Or maybe some kind of "screen-scraping" techniques to pull the data off the page... hmm...
I guess because his stats are about 2-3 weeks behind, it would indicate that things are leaning towards the manual procedure...
Spam is unsolicited, so google should filter none of his mail.
This guy solicited it.
Did anybody else notice that his site hasn't been updated in almost a month (May 25)? Seems his project is no longer working. I wonder if Google booted him.
KevG
For those of you that don't have Gmail yet, there is a little "Report Spam" button you can use to, well, report spam. When Gmail gets a few million users, and even 1% use this little button, you are going to see the spam detect rate skyrocket.
gzip it and compare the files. a short tracking code will make a negligible difference.
funny munging
If I offer 10 of my most leeaching customers 1 gig of space, I will need 10 gig of space... or will I? How much of that will be duplicated between at least two users and how much of it will be used by all 10? Remember Google already has copies off allmost all the useful stuff on the net. If you grab some random web page and mime attach it to email, thats going to waste space in my mailbox but if google can figure out that they already have all the images, as well as the text, its going to compress down to very little. For the 1st customers it requires a massive increase in needed disk space but at some point it starts dropping off. Sort of like how much stuff they have to index for the web and image searches.
Not many are as gregarious as Pratt. I've been using gmail for some time now. I must say google has done a pretty good job with their spam filters. For not-high-volume users (which most people are), gmail works much better than other email providers (i have yahoo, ureach and hotmail accounts which I use regularly).
Of course, google should improve and filter out the occasional crap I get too. And also offer 1 TB.
Science as a way of life.
When signing up for my DSL I made sure all servers were OK to run. Once I had that I setup my mailserver, learned how to admin it, learned how to run Spamassasin, gave accounts to friends/family and now I have about 10 users that hit it everyday.
So yeah, make sure it's OK with yr ISP before signing up, and then you're free to do what you'd like.
CBV
free ipod and free gmail!
No, I'm not keeping proper statistics. =b
The World Wide Web is dying. Soon, we shall have only the Internet.
It's not that bad as you think. I posted an dedicated email address to slashdot two times already, just to see what volume of spam I get. Surprisingly, it's only 2-3 messages every other day or so.
Well, I guess I need a booster shot, so here it is: slashdot@hates.ms. Spam away...
If con is the opposite of pro, is Congress the opposite of progress?
I have Mozilla, it has a Bayes SPAM filter. Lately, it's been getting fooled more and more. The messages that make it through have one or more of the following features:
.GIF form only - no plain text.
1) Several intentionally mis-spelled words
2) Lots of text in white (so it's invisible or nearly invisible)
3) Message in
Could you add filters that look for, say, more than 10% of the words mis-spelled, text font nearly equal to background color, or no actual text in message? These would take effect in addition to the existing Bayes filter.
A goal is a dream with a deadline
Which is why my email address is llllllllooong. 20 characters before the @ sign. I don't post it anywhere and I think 20 chars is outside the range of such brute force methods. I've been spam free for about 2 years. And I mean spam FREE. I get NOTHING to my junk mail folder. Its marvellous!
I tried to do the same thing with my AventureMail account but AventureMail wasn't cool with it. They deleted my account! You can check out what little data I collected before the account suspension and read the emails to and from AventureMail about the merits of the account suspension at http://3fingersalute.net/aventuremail
FoundNews.com - get paid to blog.,
Here is a discussion from Yale's LawMeme on the legal ramifications of Prattboy's experiment. Does asking others to sign you up for spam count as an opt-in?
FoundNews.com - get paid to blog.,
Right, and my Thunderbird Bayesian filter catches all of those word salad approaches. But they've come up with a new one - what I call the "encyclopedia attack."
What they do is copy an encyclopedia entry and put it at the bottom of their spam. The thing is usually a few paragraphs long, so that textually it dominates the message. The subjects are fairly random, and are occasionally educational ;)
The problem is that the text of this doesn't trip the "too many strange words" flag that's used for word salads. My Thunderbird filter is really having trouble with these. Anyone else having trouble with these spams?
I've had a gmail account for almost 3 months now. In the first month I got 3 spam messages, they all made it thru the filter. Since then I've gotten 5 more, only 1 of which made it thru. It's not statistically significant yet, but to me it feels like the filter has improved. I'm already up to 5% of my 1gig too...
What about vetting at least the image-based spam for checksumming? Scan the e-mail for image links (or images included inline). If there's a link, check it against the known list of spam links. If it's in the list, mark the message as spam. Spammers will quickly figure that trick out, though, so step two would be for Google to follow those links, and retrieve the images. Run a checksum of the image file itself; if there are a lot (say, a thousand) messages including the same image, tag it as spam. This combines spam filtering with the fun of reminding spammers that Google has an order of magnitude more bandwidth than they do. Use their own messages against them: the more you spam, the bigger the Slashdotting (Googling? Alas, that word's already taken.)
For bonus points, keep the downloaded images in the Google cache; keeps them available for the mail user, alleviating the load on the sending site for legitimate messages, and keeps them available for, well, the Google cache.
"Make it ten--I am only a poor corrupt official."
--Captain Louis Renault (Claude Rains), Casablanca
I have 3 ideas that may overcome spam.
This may require an overhaul of the email system though. One may be to have multiple addresses bound together. So you would give one email address and only "authenticated" or approved contacts could get your second address. Now sending an email simultaneously to the two email addresses would result in the email being delivered directly. mail sent to only one of the 2 addresses would be delivered as per normal, and would be subject to the normal filtering. But i guess the spammers would find ways to get both addresses too and defeat that, but it sould be doubly difficult, if not actually an order of magnitude more difficult. How may people get the same spam on different email addresses? This could be useful.
The other is to hit spammers where it hurts, audience. By rolling out a proper ad delivery system (yuck) which was separate from email, if people used their email less for getting information about products, but had it collected by some RSS type system, the spammers would be left with a dwindling audience unless they switched too. The ads would be strictly opt in.
Or mail collection rather than mail delivery. If people collected mail rather than got it delivered to them, they could in theory just not collect spam. why would anyone collect spam?
Lastly is education. if people kept their own whitelists of approved mailers, they could in theory get rid of most spam by keeping good whitelists.