Slashdot Mirror
SpamArchive.org Launched
An anonymous reader writes "SpamArchive.org has just been launched. SpamArchive.org is a community resource that provides a database of known spam to be used for testing, developing, and benchmarking anti-spam tools. The goal of this project is to provide a large repository of spam that can be used by researchers and tool developers. In the past, there were a few small personal spam archives that were used. There was no large set of spam that could be used to test new anti-spam algorithms. Thus, developers could not sufficiently test their techniques across a range of messages. Also, the lack of a "standard" sample of spam made it difficult to effectively benchmark anti-spam tools."
Do they have a mailing list I can sign up for if I want to get updated by e-mail?
I should just gzip my mbox and send it to them. That'll give them years of research material.
Kan jeg få en pils, vær så snill?
Whoever wrote this obviously doesn't have a Hotmail account.
Even I know how to buy a domain name and write a few paragraphs of text on a white background. There is nothing about this archive to hint at its origin or credibility. This is a /. worthy story?
Can't researchers just set up their own hotmail account?
Seems cheaper.
...where wizened historians wearing horn-rimmed spectacles will sit, hunched over computers, studying the archives of ancient spam.
"This one mentions sex... apparently, sex was a preoccupation of the early twenty-first century..."
Honey, I shrunk the Cygwin
Asking for a slashdotting is one thing, but asking to be an archive for spam is another.
I wonder if anyone knows just how much of the stuff is out there, and if it's even possible to store all that. Of course, spam being mostly duplicates and all, maybe they have a chance. But with spammers staying ahead of the game and rotationg their text, I wouldn't count on it.
On the other hand, why not just set up a couple of hotmail accounts, bait them a bit, and just watch the spam come in? Why even bother asking for it?
Now that spam is so collectable, someone should start a service to let people trade it?
What will someone give me for my rare "Help fund the freedom fighters in Chechnya!" complete with numbered bank accounts to send donations to?
but what use will it be if the anti-spam tools it helps develop can't adapt to new forms of spam. It is a good idea to build an archive of old spam, but what about the new spam that it will ultimately give rise to. Just like any biological system they will adapt or die. Hopefully DIE! But if not then they will be more annoying than ever.
I think that they should send email out to everybody describing this great service!
Those who sacrifice security to condemn liberty deserve to repeat history or something. - Benjamin Santayana
Dude, i could have registered a simlar domain and put up a comparable web page within a matter of hours. I hope they really exist.
Wouldnt it be great if the submit email address was forwarded to someone's ex girlfriend? Thats the ultimate form of revenge...
1) Register domain name.
2) Put up web page advertising some kind of anti-spam database.
3) Forward all email sent to the submit address to someone you dont like.
4) Get slashdotted.
The end result is that three million people send 100 spams the first hour to the submit address. Within a short amount of time, your foe has 300 million emails in his/her mailbox. Now that's spam.
Kan jeg få en pils, vær så snill?
Damn!
:)
And there I was thinking they were creating a historical archive of all the funny worthless spam we get in our mailboxes every day...
See that could turn spam in to a fun thing! set up a site where spam is ranked most popular by the number of people forwarding in the same SPAMS they get.. i think it would be interesting to see a daily/hourly/weekly TOP 10 SPAM in the world graphs..
I would do this myself.. cept i suck at html.. anyone need a VoIP network built?
With some people already accusing bugtraq of being a repository for exploits that anyone could use for exploit purposes, you'd think that the same could happen to the spam archive.
Soon we'll see old spam being recycled as the new breed of spam trolls mine the archive for inspiration - and maybe just material reuse.
Then, of course, it's not like we don't see recycled spam anyway, so maybe this isn't such a bad thing...
(And if I sound incoherant, it's 2 in the morning. I should be sleeping.)
Combine that with posting to some anti-spam newsgroups with their real email address, and bingo boingo, all the spam in the world will come right to them.
This site also creates a problem in that only the spam posted to that site might be used for research. There might be millions of spam emails overlooked because they don't make it onto that site. Think of those poor spammers that won't get filtered :)
Won't someone please think of the children!?!?
Imagine a b....
Oh forget it
"Do something man. Right now."
NANAS, or the newsgoup news.admin.net-abuse.sightings does just this. It is a public archive of spam which can be searched e.g. with Google Groups:
http://groups.google.com/groups?group=news.admin.n et-abuse.sightings
Why reinvent the wheel? Or does this new spam archive have any new functionality to offer?
Now Spam Radio got an archive to dig out new infomercials from. :)
Well, there is already a pretty large Email and USENET Spam archive at the NANAS (news.admin.net-abuse.sightings) newsgroup.
You can check the Google Groups archive
You can read the NANAS charter at http://www.killfile.org/~tskirvin/nana/charter/nan as.html
Most obvious is that usually spam is personalized, that is the recipient's mail address (or part of it) often appears either in the subject or in the body. So will this archive store every variant of every spam, or just a 'global' model ?
... ok, neither did I.) might be listed as a spammer so, there should be some re-occurrence filter to ensure that a given "spammer" doesn't send a given spam-model more than once to more than once recipients but here, once again, we may face some situation where everybody could be hurt by such restrictions.
I guess this could be easy to implement some "almost identical" recognition filter but the problem would be that somebody forwarding a funny spam to somebody else (hey, haven't you kept your very first "herbal alternative to viagra spam" spam message in order to show it to somebody ?
I personally consider the spam problem as overhyped as it doesn't take me more than 15 seconds a day to eliminate unwanted messages.
I have more problem in real life with these advertisers who dump their pizza-prices in my mailbox but here, in Switzerland, every one pay for every garbage he dumps.
Trolling using another account since 2005.
I've owned spamarchive.com for ages.
Want it? - I have no use for it.....
says:
Domain Name: SPAMARCHIVE.ORG
Owner, Administrative Contact, Technical Contact, Billing Contact:
Guru Rajan (ID00024772)
11475 Great Oak Way
Suite 210
Alpharetta, GA 30022
us
Phone: +1.6789699399
Email: guru.rajan@ciphertrust.com
http://www.ciphertrust.com introduces itself as:
Protect Your Email Gateway
Anti-spam and email security for the enterprise
CipherTrust has integrated defenses for all email application-level threats into one, comprehensive device. Our IronMail appliance protects enterprise email systems such as Microsoft Exchange, Lotus Notes and Novell GroupWise against viruses, spam, and intruders, and provides message privacy and policy enforcement.
+1 Informative? WTF? The parent is a _TROLL_, man. Look at some of those names a bit more carefully.
Exactly the opposite is needed for work on mail filters.
Spam is really easy to find, everyone knows that, create a hotmail account fill out some web forms, post to some newsgroups, put a mailto: on a web page. Wait a little while. Bingo, lots of spam.
However, non-spam email is harder to find. Using your own makes techniques that work with your particular type of email and not other people's.
Non-spam is harder to collect. Since email is often private in nature. Removing identifiers from the headers is easy enough, but the body also can contain things like addresses, emails, phone numbers, comparisons of the boss to bacteria, etc.
A collection of real emails, from which personal information has been replaced with fake data would be of great use. A few people I know are working on creating such a data set of email. It is aimed at more general email filtering though, not just spam detection, and hence requires categorisation. And is from academia and hence will probably lose the race with the heat death of universe for completion.
I do note they have a 'non-spam' heading on the very sparse web page which is encouraging.
Would spammers try to "anti-spam" the spam archive by submitting billions of perfectly normal emails?
Ian
I just added a rule to my spam filter to forward all messages!
This worthy effort needs funding to keep it alive. I have some contacts from Nigeria who may be able to help, I will forward their details.
Actually, I can vouch for this; it's totally real. I also saw it in last month's issue of Wired, list & all. Yes, I'm a bit ashamed to admit that I read Wired, but, hey, what're ya gunna do? The article on spam was really interesting and is worth a read, even if you already consider yourself an expert on the subject.
Archive of samples of non-spam messages should be collected as well, containing real E-mail messages which aren't spam. These messages should be more or less normal private E-mails which are just volunteered to make public for testing purposes.
The purpose of the samples of non-spam messages would be to help preventing false hit testing for the spam filtering algorithms, just as real spam messages are used to tune the algos for detecting spam.
--
this article reminded me of that hilarious 'spam' song by save ferris and i decided to dig out the lyrics. if you happen to know the direct url (google helps there), it works just fine, but check out what happens if you click at their link in their lyrics listing for 'save ferris'.
but what do i know, i'm just a model.
All generalizations are false
Take the test and find out... ;)
What???? I would like a link to that story just to confirm this...
Craig McPherson of the hole Debianits is an evil spammer... I refuse to believe it...
Damn back in the days on LNO he used to be such a nice troll, he had style..
- Lovechild
...spammers use the anti-spam tools to create spam that doesn't trigger the automatic spam filters.
The archive could give them a lot of valid email addresses...
Consider this one: You forward a spam to submit@spamarchive.org. The forwarded mail is now a part of the archive. Spammers snoop the archive for email addresses.
Interesting, Informative? A 4? For a troll's in-jokes?
Bah, I say.
deus does not exist but if he does
I can send them a copy of all the awesome, truly fantastic offers that arrive in my mailbox? =)
;)
Oh, the joy! 300 copies of "make money fa$t", "enlarge the size of your penis" and "Amazing investment opportunities", delivered lovingly every day to this archive, to be preserved for the good of humanity forever more!
(Clicking hysterically on the "forward" button...)
The right to offend is far more important than the right not to be offended. (Rowan Atkinson)
The parent is a troll, folks. This same email list has been posted to multiple discussions, probably by the same loser. I'd really like to see moderators show a little bit judicious. A quick search on wired.com turns up nothing looking like the supposed article. This is completely fake and some of those names should look familiar (but not for spam). Will someone more reasonable please mod this one down?
His page of graphs shows the exponential growth of spam over the past few years.
Of course, it would make filtering easier too.....
See my journal, I write things there
Aside from all the bashing these guys are getting here for not having any working code, this kind of database would actually be quite a good idea.
One main problem for anti-spam is this: humans are very good at telling spam from legitimate messages. Comupters are nowhere close. Why not? Well, humans are simply better at certain types of problems like pattern recognition because of centuries of evolution. But there are ways around this: genetic algorithms and neural nets are two that I can think of. Both of these are "learning" strategies and need large databases to get started. We're talking about billions of messages or more, not the hundreds that you get everyday.
So the kind of database (one for spam, one for non-spam) that these guys are talking about would be an excellent way to develop intelligent spam-detectors.
Sorry if this is unpopular opinion, but we are against legal and in favor of technolgical solutions for most of the problems of the internet, aren't we? Then why are we waiting for anti-spam legislation to fall like manna from the sky? The best way to fight spam is using technology. Methinks this is a step in the right direction. So get off your ass and contribute. Forward your spam to them. Think of clever algorithms that can make good use of a large database. And code them. And submit patches. Isn't that what open source is for? Hey, may be this is going to be a killer app for open source, considering how big a problem spam is going to be in the next few years
Anti-Spam Community Links
check back as we create a resource page for the anti-spam community.
Exactly what's the definition of a 'launch'?
Geekiness has reached a new high! Or should that be low...?
Does this make my brain look big?
You guys are a bunch of frickin' sheep.
If anyone writes an anti-spam tool, I need to distinguish between spam and non-spam, making non-spam equally valuable for spam-filter benchmarking.
Having a log with only spam makes it quite easy to achieve a 100% benchmark (simply reject it all!).
Couldn't find anything about this on the site, so unless I'm missing something, the value of such a log is limited at best.
This isn't like Distributed Checksum Clearinghouse or some other spam *solution*. It's intended to test to see what percentage right antispam tools get right -- false positives and negatives. It's useless (at least directly) to end users.
So unless your antispam tool breaks on some names in personalized letters, I would think that it's okay.
May we never see th
Hey, lovechild!
I was about to add a comment in the general idea of above post, but he pretty much sums it up.
Having a 100% spam archive is pretty useless as a base-standard for doing tests. U need to have at least as many REAL mails (I made up the amount,but u get the idea) for a base to work from.
It's easy to identify all mail as spam, it's much harder to identify the real mail in between. This is probably a different approach as well. Rather then looking for spam, you can try to filter out real mails.
Do not reply to this with technical stuff. I am not THAT technical, and probably the latter suggestion I made is not technically feasible.
But anyway the main point is: a 100% spam archive doesn't seem that usefull to me..
You might as well start up a database to catalogue all the different shapes of sand on the seashore - largely useless exercise in futility.
What people are starting to do is block EVERYTHING that isn't on a 'whitelist'. That way granny and Junior don't get mail from anyone unless they're pre-approved. If they get mail from J.Random Stranger it's bounced with a request to put a short random token in the subject line. Thanks to marketing a good third of Internet mail traffic is useless crap. Thanks marketers!
To show just how evil and desperate unemployed, cash strapped, deep in debt spawns of satan those people are - yesterday I got a letter from my mortage holder, Chase Manhattan bank, marked "IMPORTANT ACCOUNT DOCUMENTS ENCLOSED". It turned out to be yet another credit card pitch. ("You qualify to give us even more money!!") Bastards. It's not my fault the Msft office automation vision they bought into turned out to be way more expensive than the sales flak led them to believe.
I wish unemployed marketers would turn to prostitution and drugs instead of spam - at least they'd be supplying things people actually WANT.
try { do() || do_not(); } catch (JediException err) { yoda(err); }
MOD DOWN PARENT!
McPherson, Craig, doesnt look like a spammer - I remember a couple of years ago at LNO. He's a decent troll.
trollmastah@hotmail.com - Really, one of the countrys top spammers with a hopmail address?
*@adequacy.org - that well known site isn't spam central.
It's a trap!!!
1) Set up story about new site accepting spam to assist in creating better anti-spam tools.
2) accept all the submissions from the teeming millions(tm) at a popular tech site or two.
3) cull all the email addresses from those duped to forward spam to you.
4) sell said email addresses to spammers.
5) PROFIT!!!!
would be simply posting the email address here, and on alt.sex.*
then let the email scrubbing bots do the reat.
Call me a cynic, but in my estimation, the only thing effective Spam filters based on content are going to do is make Spam more annoying. Why? Because spammers are going to have the same access to filters that regular people do. All they'll need to do is run their Spam through the filters to check and make sure they pass. In other words, if these Spam filters really work well then it won't be possible to determine what is and isn't Spam by a quick glance at the subject line or formatting of the message. Rather then "INCREDIBLE OPPORTUNITY FOR FAST EAZY MONEY$$$$$$$$$5390ANFP9O" and "HOT HORNY SLUTS WANT TO MEAT YOU" we'll get stuff like "Dude, check this out!" with a body like "hey man, long time no see. What have you been up to? I've just been hanging out, not too exciting, although I met this cool chick off the 'net. Hrm, you still looking for a gf? You should check out FriendFinder.com :). Anyway, talk to you later, bro."
And you'll need to read the whole message before you realize its Spam
You might not like to believe it, but spammers (or at least some spammers) are hackers, in both senses of the word. ESRs supposed "hacker ethics" are as much bullshit as anything else he says.
The only way these things will work is if the vast majority of people do not use these things. I don't know how likely that will be, with MSN already promoting it's 'less Spam' features.
I think what we need is a fundamental change in the way email is handled. The current system is just way to prone to abuse, and should be replaced entirely. The new standard could use things like digital certificates and other technology to make sure you're talking to an individual (while protecting anonymity in some cases, although the receipt of anon email could be optional, etc, etc)
autopr0n is like, down and stuff.
If you were a spammer and wanted to collect a large number of valid email addresses, how about this as an idea...
1) Produce a website pretending to be antispam.
2) Ask people to send their spam emails to the site (generally including a valid from address of course)
3) Publish on slashdot so as to get lots of interest.
4) ???
5) Profit!
(Unfortunately, we all know what stage 4 is for spammers...)
wot no sig
Think about it: while 99.999...n...9% of spam mails are either deleted before they're read or shunted into a "Spam" folder, there will be enough Internet newbies / technology imbeciles / other non-slashdotters ;=) who think that unsolicited emails can be a cure to their debt problems / small penis / whatever.
So long as enough people are suckered by the adverts, the spammers get enough to pay their bandwith bills, and they can continue to spam us.
What's needed is education for the naive: just ignore unsolicited adverts. TOTALLY. I mean, when was the last time you opened a credit card mailshot? Or one of those "Espescially for you" things in real life?
Exactly. Trial by error is not a good learning solution for spam. It should be mandatory that all ISP sign-up procedures inform new customers that any unsolicited emails can safely be ignored, hopefully that way the spam industry will start to wither and die.
-Mark
it is exactly the same thing as www.spamrecycle.com that exists for a long time now?
BoD
BoD
Hi Anonymous Coward...
- Lovechild
What's the point of testing a filter against a database of known spam if you can't test it against a database of nonspam?
Anybody can write a filter for bulk mail. How do you differentiate between solicited and unsolicited bulk mail?
Take piracy, for example. As soon as someone finds a new way to stop people copying games someone else finds a way around it. It's been going on for years, and it's unlikely there'll ever be a way around it (even Palladium will have its holes).
Today, for example, I received a spam inviting me to "increase the size of my *enis". They are obviously aware that the word "penis" is blocked by many mail systems so they simply found an easy way around it.
Sorry, but my karma just ran over your dogma.
Great, Theses lasts times my inspiration has felt... With this great source, I could take some good example for my next piece of spam :-)
Hey, again, Lovechild!!
I discussed this idea yesterday with my manager. I've been looking at spamhaus over the last couple of days but they don't take spam reports from end users. So I had the idea of setting up a domain for users to forward spam. This spam database could then be used to create an RBL for the most active mail relays. I suppose now I can create the RBL without collecting the spam. :-)
-- Thou hast strayed far from the path of the Avatar.
cynic
Don't know about you, but I check the sender before I read it, and quickly scan for any URIs, if they seem for me. I don't know of anyone who would send me something like "check this out" anyhow, not by email, that's what IRC and IMs are for.
"If anyone needs me, I'm in the angry dome."
How does this work, you ask? I create a new email address each time I give out my email address. We have a sendmail setup that allows you to make "username+foo@example.com" go to "username@example.com" where "foo" is any arbitrary string.
So, amazon.com thinks I'm "username+amazon@example.com", securityfocus thinks I'm "username+bugtraq@example.com" and so on. Once I receive spam on one of the addresses, it's trivial to write a filter that matches with near 100% confidence ("username+bugtraq@example.com" should only receive messages originating from securityfocus, etc.). Most times, if an address receives a spam, I can just procmail all mail to the address to /dev/null (eg, no complex rules like for the bugtraq example). This also allows me to track where spammers get their lists.
We use sendmail. Equivalently, qmail allows "username-foo@example.com" and if you own your own domain, just use "foo@example.com".
I find this advanced filtering stuff fascinating, from a completely academic point of view. I, of course, can't apply any of it since I don't receive any spam, but it's interesting nonetheless. I just read through how the Bayesian filter works. It is very simple: it only filters based on word (token) probabilities. So, it would assign a value to "make," "money" and "fast," but not "make money fast". Seems like you could get much better results if you do something more advanced like Markov chains or a neural net. There's lots of research out there on textual matching, and I'm not sure why people would start out with such a simple algorithm when there may be better things available (where "better" is measured not only by accuracy, but also by training time).
Spam, like all written text is subject to copyright
Couldn't the spammers sue for copyright infringement?
According to WHOIS, "spamarchive.org" was registered by one Guru Rajan, who has an email address at "ciphertrust.com". Also according to WHOIS, "ciphertrust.com" has the same person as technical contact and if you check the website you find they are the vendors of "IronMail: The Secure Internet Email Gateway", an established if not well known product.
In short, yes, it seem legit, and it probably took me less time to find that out than the time taken by the myriad people asking "is it legit" took to post the question. ;)
UNIX? They're not even circumcised! Savages!
BTW isn't this what is called a 'corpus'? You can do some beautiful research with it...
Pfhu! that's just low quality run of the mill bulk mailed average sort of spam. I however am in possesion of some serious l337 spam that's just not sent to anyone, and I'm not parting with those lumps of resercher gold, oh no.
No free lunch for you.
They've got gazillions of messages sent to uce@ftc.gov
Why not just make that available to the public for creating training sets for spam?
The idea of a central archive is good, but I don't see why there's a need to reinvent a New! Improved! wheel.
We set up a quick website, tell people we are 'collecting spam' (make up a good excuse) and voila! thousands of _verified_ email addresses, belonging to well connected people with high incomes within hours! Thanks Slashdot!!
AC
I don't see how this can work. Sure, hard drives get cheaper all the time, but how can they possibly afford to keep up with a wide open "send us spam" request? They'd need petabytes of storage.
that image of a pig merged with a block of spam is quite revolting! Please change it.
...nicely burnt into a DVD. Provide the extraterrestrials with a realistic sample of our culture. :-)
They won't ever bother to contact us any more after that... or they'll zap us in an instant. Either way the world will be a better place
"spamarchive.org" is registered by "ciphertrust.com".
Ciphertrust develops and sells spam-prevention software.
Interesting.
In order to counter the rising tide of spam I recently installed a spamblocker, even though I'm wary of such beasts because of the danger of false positives.
:(
Sure enough, I have received false positives. But only from one source: my filter traps the Network Solutions email asking for confirmation to proceed with the transfer away of a domain to another registrar. Net$ol changed the format of these emails a while back: they now start off by talking about a "special offer" and it's only towards the end that the real purpose of the message is revealed. My suspicious mind wonders whether these emails are intentionally designed to look like spam to reduce the number of successful transfers... sneaky
Do we know that this is a good site, or is this a devious mechanism to collect the email addresses of everyone who forwards them spam?
Gimme a break, they could have just set up a hotmail account and left it for a couple of hours.
Sounds like an overenthusiastic noob just got himself a domain name.
Any email sent to me that calls me 'dude' gets itself automatically deleted...
Oh, it must be true then! :p
As anyone knows, filling webform as required by some sites leads to the snail spam being sent. Some of these forms are prompting to fill in manually the position you are in. Usually I omit these as they are mostly non-mandatory. However, once I put myself as 'Yellow Snow Developer' and, lo and behold, here is the envelope in my mail addressing me as such.
Let them entertain the community (and a few post workers)
This reminds me of an idea that i've had for som etime.. spamnewsreportingforthemasses.com - A news site reporting news from spam-sources - sort of like a satirical view on spam.
"New indian health care enables you to have more lovers"
"New solution for your economical problems found"
- and throw in a hoax section too...
The domain is registered to Guru Rajan of ciphertrust.com. Funnily enough, Ciphertrust markets a product called IronMail that does (among other things) spam detection. So who says they are really putting the database out once they have it and not use it for their own good?
Tired of spam? Get advanced junk mail protection with MSN 8. http://join.msn.com/?page=features/junkmail and depending on where the sender lives it will spam you in a different language, Direct chatten met je vrienden met MSN Messenger http://messenger.msn.nl
How lame is this? A simple domain just registered, nobody has a clue where this idiot starting it came from and a simple static html page.
This made slashdot headlines, because!?
1) Open Hotmail account
2) ???
3) PROFIT!!!
how does one change his
Although spam eradication is a good idea in general, I wonder if bulk training will only result in resistant strains of superspam developing, much like the v-cillin resistant staphs that are popping up lately.
If we deal with a little spam by hand today, will that keep us from having to deal with undetectable spam later? I can imagine spam systems that probe you (using actual system probes of you and your contacts, marketing history and social engineering) to target spam that you may actually believe is a recommendation for the Sony(tm) handicam from your Uncle Bowser, or really is your wife asking you to pick up some Clorox(tm) brand bleach and fabric softener on the way home...
Luckily, neither of them is likely to be sending information about my penis to me at work.
Much like modding the Xbox (and thus giving MS the practice they need to harden Palladium), giving the hard fight to the spammers might just backfire on us.
Can someone who uses Frontpage to create an anti-spam homepage, be trusted?
not too much work.
who's moderating the meta-moderators?
spamarchive
I expect we'll next see Spammers using the DMCA to get their copyrighted SPAM removed from the database...
Andrew Semprebon EQ Systems Inc.
I told my wife that I was going to make a deposit at the spam bank. I went like 5 times a day.
She's amazed and grateful that I can even 'log in' any more.
Hey, that wasnt me!
Are they going to offer the content of spamarchive.org under an Open Content license, or is this just another database that will eventually be absorbed and closed to the public by some corporation protecting database copyrights?
--LP
You got the joke!
Rich.
libguestfs - tools for accessing and modifying virtual machine disk images
Never hit your grandmother with a shovel, for it leaves a bad impression on her mind...
Fighting spam is like fighting crime, hackers or piracy. For every measure we put in place some spammer somewhere will find a way around it.
All problems are not the same - some have solutions and some don't. Take spam and piracy for example.
There's a system out there right now for spam blocking (I forget the name or URL at the moment, but it's been mentioned before on slashdot) that maintains a whitelist of people that are allowed to contact you, and when it receives an email from a person that is not on the whitelist, it stores that email in a temporary area and emails the sender asking for a confirmation email in return. If the spam-blocker receives a confirmation email (i.e. the actual person gets the return email, hits reply, and hits send as per the directions) then the original email gets through to your inbox. Right now this is a 100% effective spam-blocker. No good email is filtered out, and no spam is let through because spammers forge their return addresses and therefore never get confirmation emails. It has the added bonus of not requiring the user to look through a "junk mail" folder. Implementing this system universally (1) server-side would solve the spam problem. The only way spammers could get through would be to provide actual "from" email addresses which open them up to lawsuits, and (as they have to check incoming messages and reply to them, meaning they have to either host the "from" account themselves or have fast access to a server that does) it would open them up to all sorts of DDoS attacks. Got a 1KB spam email that slipped through with a from address of from@spammer.dynamicdnsservice.com? Hit that ever so satisfying "Can The Spammer" button and blast spammer.dynamicdnsservice.com with 100KB of data. The more spam the spammer pushes out, the more clogged its downstream pipe gets.
(1) Ok, not this system, as a spammer could always find out who your friends are and put their email addresses in the from: header, but a system based on public key cryptography would do the job nicely. That would mean client-side software updates and a protocol change, but it's still a solvable problem.
Now, take a look at piracy. There is a property of information (or data, or bits, or whatever you want to call it) that is so absolute and inviolable that I would go so far as to call it a law of the physics of information. It is: The only way to control the distribution of information is to ensure that the people and machines that have access to that information all agree to control its distribution. That's it - think about it. It means every technology-based digital restriction mechanism can be broken. (2) Yeah, you could put telescreens in all homes and watch everyone 1984 style, but that's a very poor solution. The best way to deal with "piracy" is to stop thinking along the lines of trying to control information like a physical good and find an alternative business model. No endless wasteful competition between DRM designers and hackers, and no more buying expensive DRM snake oil for businesses.
(2) Yes, even palladium can be broken. Here's an easy three-step process for breaking a palladium system:
(1) De-solder the TCPA components from the motherboard except the CTRM (yes, including the cpu if necessary), attach them to an add-in pci card along with a power connector (again, if necessary) and a pci interface chip that talks to the bus and simulates a CTRM that has "measured" a trusted system.
(1.5) Not really a "step". Design and fabricate the above chip.
(2) Write a kernel level driver for the OS of your choice that diverts calls to the trusted hardware subsystem in loaded applications to calls to the driver itself which simulates the trusted subsystem. Any time it needs a "Yes, I am a trusted system." certificate signed, the driver should call upon the pci card to perform this function. (Yes, you can install your own drivers. You just have to boot your system in untrusted mode [where applications would normally not receive services from trusted hardware])
(3) Download "protected files" and let your trusted applications happily place them (in encrypted format) on your hard disk. When you want to directly access the unencrypted data, snag the decryption key directly from the driver.
Yeah, it's complicated, and not all people have the necessary skills to pull it off, but keep in mind that:
*It only has to be done once to release information from DRM jail and make it available to anyone.
*Once the step 1.5 chip has been designed and the driver written (along with a userspace "data recovery" tool), they can be sold fairly easily as the equivalents of "mod chips" in game consoles.
Two last important notes:
*Yes, I've read the TCPA specs and I know this will work. If you would like to verify this for yourself (a smart move), they're freely available for download in pdf format from the TCPA web site.
*This does not mean palladium can be safely ignored - quite the opposite. When the only legal way to access certain content and services is an attempt to violate the physics of information by a single convicted but unpunished monopoly, everyone is in trouble. I'm sure you can think of other terrible consequences, but here's something to get you thinking in another direction. What will happen when everyone trusts the "Trusted Computing Platform Alliance" enough to put their personal (medical, financial, etc...) information into the system?
because spam has 'evolved' into a more dynamic form. Spammers now append random letters and characters into emails so they don't match theese filters. So then the filters 'evolve' to match the patterns of randomness. It is an evolving game back and forth. But all in all it remains simply pattern recognition and pattern generation done by computer. If a computer can make it then a computer can un-make it. So really we will never be thru with spam until we take a less 'trusting' approach to email. By that I mean we all have to adapt to a more opt-in method of reciving mail.
Tagged addresses (na+foo@example.com, "foo" is the tag) are automatically routed to the correct Email address (in the case of my example to na@example.com).
There is no need to set up any new email addresses.
I use it all the time. Too bad many online vendors do not allow me to enter the '+' sign on their registration forms.
Proletariat of the world, unite to kill spammers
In Soviet Russia, I ruled you
I surely hope this project gives developers a great tool to fight spam, and I wish best of luck to them! But there's still that deep feeling in me that says that I will miss my ~100 spams a day. I was so used to see messages like "MAKE MONEY FAST!" that I think a really (or nearly) perfect spam filter will make my inbox so empty... :'(
Gotta delete 'em all.
Maybe they should rename it CanofSpam.org? ;-)
koolaid
I don't thing that a large archive of spam is hard to come by. You don't need to publicly invite submissions either - just acquire a domain and hosting with catchall e-mail service, set up e-mail forwarding to an address for your database, then publish several addresses under that domain where spammers are bound to pick them up (newsgroups, FFA lists) and register them with services who sell their e-mail lists with a lot of different demographic information vectors. You'll get as much input as you have a use for.
For calibrating spam filters you'll probably only want spam from the last few months as spam does evolve - e.g. it's mostly herb*l vi*gra these days.
What is at least equally needful but much more hard to come by is a large, representative collection of legitimate e-mail, to test spam filters for false positives. This collection would need to cover diverse languages, cultures and contexts (private, business/x-industry, business/y-industry, system error messages, automatic notification messages etc.)
What is hard about this collection of legitimate e-mail is that the privacy of both sender and recipient is affected, and that, if confidential information is masked or deleted, the e-mail isn't the original one and spam filters might evaluate it differently.
There is one subset of legitimate e-mail available: public archives of mailing lists. But these e-mails don't cover the style of e-mail in other contexts.
Moreover, as far as I know most of us, when teaching our relatives to use e-mail (hey, the revolution has to propagate somehow) we also teach them how to quickly spot and delete spam. That is, we teach them to think "Joe Computer Professional said it would be better for all of us if I just ignored spam". But if filters are installed everywhere, spam will become a different beast - spammers will write message that get through filters and are thus inevitably also harder for himans to distinguish as spam. Because of this more people will read it and also more people will buy things from spammers. Ergo, spam will become more profitable because filters will force spammers to be less stupid.
And I know that this is not an impossible prediction. Thanks to a little care with my e-mail address (and I mean a little - I don't even have a scratch account for use at the less reputable parts of the 'net I visit), the fact that my main account is not with a big provider and probably most importantly SpamAssassin, I receive very little spam - less than one messsage a month. But I once got a rather long message that I had to read through twice and visit the URLs given in the message because I couldn't figure out whether it was a legitimate mail, bogus randomness by an insane businessman or true spam. I wouldn't want to have more of those taking my time.
... and I receive zero spam
...
Once I receive spam on one of the addresses...
I also advertise the email address widely
So, you receive no spam, but when you do receive spam, you edit procmail. Which is it?
Also, you widely advertise your email address, but you don't actually use your email address, but made-up aliases. Which is it?
You're simply masking the problem, and going thru a moderate amount of gyrations (which most average joe 'net users won't/can't go through) to do so.
creation science book
Cloudmark makes SpamNet, a P2P plugin for *gasp* Outlook, that allows users to submit spam messages to a database, where an algorithm integrates the submissions into a master spam list that gets published back to the clients, which then pull messages out of users' inboxes as they arrive. (Works pretty well, too.) I should think their DB would be a good place for this effort to begin.
Another idea to catch up some spam is to write a message in a newsgroup, using a real email address.
Surely you'll fill the mailbox quickly.
I already _have_ a large repository of spam in a set of folders
in my mail repository. The US FTC already has a _huge_ repository
of spam. The news.admin.net-abuse people have a positively
*enormous* repository of spam from both email and usenet.
Anyway, a large repository of past spam is not really what you want
for testing anti-spam solutions, because spammer tactics keep on
changing. It used to be that a whitelist solution could trip on
unrecognised From: fields, but now they're using the same From:
field for everyone. It used to be that you could filter by the
IP address of the mailserver used, but these days the mail servers
migrate constantly across entire Class B networks. It used to be
that you could filter based on subject lines with lots of digits
at the end, but these days they're using random sequences of
letters, and if you filter based on that they'll switch to Markov
chains, which are simple to create and AI-complete to recognize.
For anti-spam testing, what you want is a mail account that never
gets anything sent to it _except_ spam, for which you can create
infinite alias addresses and release them in strategic places.
(You start by designating addresses starting with u as having been
released only on usenet, generate a few thousand addresses that
start with u, and use them in the From: fields of a bunch of posts
to test newsgroups.)
You have to be constantly getting _new_ spam for testing. The old
stuff will give you a false sense of how well your stuff is working.
Cut that out, or I will ship you to Norilsk in a box.
Who do you think this will help most? The people making anti-spam software, the people sending spam, or the clever ones that send spam telling you to buy anti-spam software?
Get your now! You gate to betta rife. Moa pay, wok wess.
www.dipwomas.tw
I am very small, utmostly microscopic.
There is an OSS project I can contribute to !
An "standard" archive of spam might work great for benchmarking rule based filters against each other, but adaptive filters, like the popular Bayesian kind, work best when they learn on your own emails and spams. There's also no point in testing an adaptive filter when you can't also feed it non-spam emails.
Zambozay! My brain must've been eatin' a sandwich!
Ok... for the people that still use Outlook, this exact service is provided by a company called CloudMark. The address is Spamnet.com. I've been using it for some time and it seems pretty robust. A community basically earmarks spam messages and based on votes a piece of spam gets moved to a spam folder on retrieval. Nothing is ever deleted.
- Google Groups: NANAS
- Charter
- Newsgroup Public Key
- nana.* Homepage
(Yes, I know others have stated some of this stuff, but it's worth mentioning it again.)Can the spam writers claim copyright infringement?
My
Limekiller
Your tax dollars have already funded a huge archive of spam at the Federal Trade Commission. In fact, they are running out of room to store the stuff. The FTC says they can't release the contents because of privacy concerns, but surely there is a way around this: xxx out receivers' email addresses; apply secure aliases to protect the innnocent, etc.
Is there anything out there that will let you submit the daily spam you get to Razor, SpamAssassin, and SpamArcive?
Right now, I use Pine and I can: 'razor-report -d' my spam (speaking of which, I've not been getting any mail caught by Razor for the last couple of days).
If someone could tie all these puppies together so that every e-mail I receive goes through these filters, My spam would be even less.
Some of us like spam, some of us don't have a life and we feel loved by the quantity of spam we recieve. It's like mail order catalogs for your mother. Long live spam!!!
The second issue is whether it's covered by the laws that supposedly protect e-mail conversations. I've been socialized on Usenet to believe that it's illegal to publically post pieces of private e-mail. I've never seen this law, so perhaps it's merely socially condemned practice. That would be an interesting question too, to see if spam can be considered "private e-mail". Since the same e-mail is sent to millions, it probably isn't. But what if it were personalized? Would it count then? Interesting issues to spice up media law...
how long before one of the spammers sue the site for copyright infringement for making
publicly available the end results (ie spam) of their hard and honest work?
i'm willing to bet my two cents on that the spammers will win the case.
---
i'm not paranoid, just scared of 'them'.
Now if you move to a statistical method, there is the issue of training your filter. By the nature of the statistical method, it may well be more accurate if you train it, as opposed to the masses. Why? Because your pool of Ham (non-Spam) is going to have distinct characteristics that will help avoid false positives for you (but maybe not someone else). If a community trains it, then on average, it may be that the Ham becomes less distinguishable from the Spam.
On the other hand, this second point is an empirical claim. It would probably be relatively easy to do a little study of this. Get some 100 people to share statistics on their Ham and Spam (not the actual messages). The researchers see if the aggregate generated filtering is better than the individual ones. Nobody's privacy is (too) compromised.
To be usable for algorithm testing, the spam database would need to be divided into a "training" set and a "testing" set. Algorithms would need to be tuned based only on the training set, and tested on the testing set. Otherwise any stats obtained will be over-optimistic, as the algorithm might be deliberately or accidentally tuned to work really well only with the particular messages in teh training set.
Spammers:
1) Download spam archives
2) Download tools to fight spam
3) Generate new spam that doesn't get caught by tools in 2)
4) Profit
So does Anonymous Coward have good karma?
The hottest site on the net!
SpamArchive.org
--been following this spam problem for awhile. One of the ideas I have seen that seems to me to have a more pro active approach to it is to poison the spammers email lists on purpose by using their own robots against them. Instead of trying to build filters and generate lists of IP's to block and etc, wouldn't it be better to create masses of webpages that contain nothing but zillions of bogus but good looking email addy's? From what I understand it's expensive for the spammers to send out huge numbers of spam emails, the profit margin is slim. This idea might knock it to the mass-zero level for most of them as it would become unprofitable for them to be in that business. If thousands of websites had a page of bogus emails, and they were different, then eventually the spammers harvested lists would be filled with useless mostly emails and the bouncing would resemble superturbo flubber.
I'm not good enough to know if this would work or not, just seeking commentary on it.
hats were ants?
That's one cool radio station. I hadn't heard of this one before, but they use music by Monotonik (used to be Mono, with guys like Supernao and Mortimer Twang on the Amiga). Excellent 'electronic' music. The spam voiceover makes it very unreal.
Maybe CipherTrust is trying to find new ways to beat peer to peer spam-fighting software from competition.
SpamNet: join up with this innovative service and help fight spam across the Web. (Spam-Filtering Software).
Scott Parker
534 words
1 December 2002
Internet Magazine
97
ISSN: 1355-6428
English
Copyright 2002 Gale Group Inc. All rights reserved. COPYRIGHT 2002 EMAP Media Ltd.
You may have noticed spam is becoming a serious problem. Not only is it clogging up email servers and wasting our valuable time, it's also likely to be the sort of stuff you or your family don't want to see. So what do you do about it?
Well, we should all be careful about where we display our email address, and create special accounts to use when registering products and services. But even if you never give your address to anyone, spam still gets through.
There are various products designed to combat unwanted email, but Cloudmark has come up with a new solution. SpamNet is a worldwide community that aims to identify and filter junk mail before it arrives. It's a free Outlook plug-in that lets you report any spam you receive to the entire SpamNet community. It's easy to install, but you might have to tweak things to get it to operate behind a firewall.
SpamNet adds a couple of extra buttons and options to your Outlook toolbar, and creates a Spam folder. Any incoming mail identified as junk by the community is diverted there, but you can also run the service on any existing mail folders.
You can filter and report any spam that does get through at the click of a button. To maintain the integrity of the spam database, each member of the SpamNet community is rated according to how much spam they report and how accurate those reports are-so reports from long-time, trusted users will carry more weight than others. This is important, as the network is open to abuse from people trying to block legitimate email.
SpamNet sounds great in principle, but does it work in practice? We found it managed to filter incoming email quite effectively, diverting about half the unsolicited mail we received into the Spam folder. We also found that running SpamNet on an existing mail folder crashed Outlook several times, although it did eventually shovel the majority of junk into the Spam folder.
The filtering wasn't perfect--it did class some legitimate emails, including personal ones, as spam. These were easy to retrieve, as SpamNet doesn't actually delete any messages, but it does mean the odd genuine message might be missed.
The Beta release we tested is only available for Outlook 2000 or XP, but there are plans to release a version for Outlook Express, and hopefully any problems will be ironed out soon.
SpamNet is an effective filtering tool, but you still have to download the junk mail and delete it. And if you were hoping to hide the spam from your kids, think again-it all remains on your machine.
***
I would be reluctant to forward messages directly from my personal mailbox to such an archive, in case the headers of my forward get left in their archive.
/. is a little premature in posting this. The concept is great, but until some content is available from their site, I wouldn't exactly call this a "launch".
My email address would then exist in their archive, and could be wrongly identified by some developers as a spammer's address.
Or worse, my email address could be spidered so that I could be delivered more junk mail.
As has already been suggested, some assurances on this site are in order. I don't know who these people are or what they're going to do with my spam when I forward it to them. And the archive is not available to me yet.
Perhaps
Instead of testing anti-spam tools...they'd better create an online list of known spammers and let isps block their stuff from arriving in innocent peoples mailbox
I love this idea.
Among my other activities, I maintain a spam filter . Like most other people who do spam filtering, I rely upon my own spamtrap addresses, reports by my users, and then crosscheck with news.admin.net-abuse.sightings and a few private mailing lists used by anti-spammers. A canonical archive of spam, however, would be a wonderfully helpful tool.
I can see a number of issues that will need to be managed with a list like this, however. Here are a few:
This is not a trivial issue. Relying on reports of spam from random individuals almost guarantees that some of your "take" will be legitimate, solicited email. Some spammers report legitimate email as spam in order to make a spam filter ineffective by polluting it. Some anti-spammers consider all commercial email to be spam, whether it was solicited or not. Other users sign up for an email list and then forget that they did so -- lots of people are trigger happy these days because of the deluge of spam. (I'm not making this up -- this has happened to me more than once.)
However, if people submit spam sent to a spamtrap address to the archive, spammers can then access the archive and remove those addresses from their mailing lists, or "listwash" them, making them less useful. In addition, troublemakers can feed those addresses to web sites or subscribe them to legitimate mailing lists. This ruins these addresses for their intended purpose. It can also result in mailbombing spamtrap addresses with a flood of confirmation messages for properly-run email lists.
I'm sure I'll think of other concerns as time goes on, but this should get some discussion started. I can think of some ways I'd handle these issues, but I'd like to hear what other Slashdot readers have to say....
Catherine
The spamarchive only helps in testing filtering algorithms for false negatives.
There needs to be an archive (corpus) of non-spam email so that filtering algorithms can also be tested for false positives.
This is a fairly useless idea for a website, as to look at spam, all you have to do is open up your e-mail inbox. Why would someone actually care to look at spam? How bored would they have to be to take actions like this without going insane? Why post spam? Are the maintainers of the website being paid for this, is providing spam to the rest of the world overly important to their whole idea of the what information should be online, or is it just plainly that they have an unnatural obsession with the unsolicited bulk mail?
SPAM is not something that should be celebrated or thought of as entertainment. It is an annoying advertisement that has turned into the world's largest electronic nuisance. The idea just seems to be a waste of time and money, and bandwidth.
From their website:
"We will publish SpamArchive.org mailing list information soon."
So, yeah, probably. Probably gonna contain some spam too.
Hey, waitaminute, that is my spam! It is copyrighted by sleazydroid inc. I will unleash the thunder of the DMCA on you if you not immediate remove my opt-in, completely voluntary requested emails!
The US government is my friend, you are not!
So called "spam" are actually copyrighted works. Please desist in storing spams,comparing spams porperties, etc. This violates the DMCA.
A Good Troll is better than a Bad Human.
collection. I have been collecting this stuff for years now. I have enough of it to fill a CD. Maybe I can sell it on ebay. Honestly, if they want my spam they'll have to pay me for collecting it in my inbox, it really is hard work (unless your address ends in @hotmail.com). I would love to see any spammer take a loaf for the team.
Karma: Censored (mostly affected by decency laws)
Ok so there is always sex, but surely the spammers will be able to target spam at people who just have not decided to buy their product yet, and may actually want er 'info-email'.
Does anyone have click through rates and 'success' of spam?
Be Free: Free Software Tuition
One organisation? That sucks. We need a peer-to-peer distributed method for cataloguing spam...
How about we create a Napster clone that has a bunch of master server lists and 'Ultrapeers'.
No, scratch that. We could use email!
Who wants to sign up to this peer-to-peer distrubted spam catalogue? AOL or Hotmail account reccomended, buy not required.
catch (HumourFailureException e) { e.user.send("You, sir, are a humourless idiot."); }
I never read every post, but I use MailSmith (http://www.barebones.com/products/mailsmith.html) and report my spam to Spam Cop (http://spamcop.net)
Of course this is Spam I downloaded first... SpamAssassin is also pretty keen and I want to make greater use of it.
Surely SpamCop or SpamAssassin already has a pretty good database of Spam. I know I send in a pile mostly from my university account which though largely retired was in use on the UseNet, Web etc. since 94/95.
Muskie
Seeing how addresses can be harvested like in this "Story of Nadine" it might be fun to plaster a email address all over the web such as JoBob@hotmail.com which is really an alias that immediately forwards the mail to submit@spamarchive.org. Heck, if each one us set up one alias on our mail servers to point to their submission box I bet we'd fill them up with data REAL quick. I might even think it would be the first time a mailbox got slashdotted...maybe.
Life moves pretty fast; if you don't stop and look around once in a while, you could miss it. -FB
...popfile and your spam problem might disappear, however your idea of spam and mine may not be the same so it might not work as well as you'd think...
Large print giveth, and the small print taketh away
My first post on /.
Hey Gendou,
I was interested in conversing with you about the posts you made on the Tresco (warez guy) intellectual property thread. I couldn't gind a superior way than this to contact you. Please email me @ fanniecat@hotmail.com with contact info so we can IM or something.
As for these subjects being duscussed, I think spamarchive.org sucks, what a lame website! And I don't judge people for what they read; I try to get information from as many sources as possible. Wired is OK, like everything else, in moderation.
I hope people using this collection will make sure they test it against a decent sample of non-spam as well. Otherwise I imagine there could be a pretty good chance of false positives, i.e. marking non-spam as spam.
bits and peace
Nicholas Daley
The only promotion rules I can think of are that a sense of shame is to ... whenever you think
be avoided at all costs and there is never any reason for a hustler to
be less cunning than more virtuous men. Oh yes
you've got something really great, add ten per cent more.
-- Bill Veeck
- this post brought to you by the Automated Last Post Generator...