there's a nice obvious "report as spam" button on every page
Indeed every mail provider should have such an interface: a trivial way to report filtering mistakes. But you over-estimate the value of everybody else's spam reporting. A filter based only on your own reporting can have a vanishingly small number of false positives, and a small number of false negatives. So small that the total amount of reporting you have to do is no more than for Gmail.
But many appliance manufacturers promote the scenario in which the user is not prepared to offer any feedback to the filter. It is much harder to achieve reasonable error rates in this mode of operation.
Bottom line: Gmail's filter is pretty good, but not better than the personal spam filters I've tested. I have yet to see a "hands-free" solution that is as good as one that uses feedback. The amount of feedback required is trivial.
If you want to stop crime, the penalty should be, and perceived to be:
- certain
- immediate
- more costly than the benefit of the crime
"Law and order" advocates generally advocate draconian punishments, but there is no evidence that they help, beyond counterbalancing the benefit of the crime. Increased detection speed and likelihood are far more effective.
You might think that draconian punishments increase the expected cost, even with haphazard and delayed detection, but they don't increase the perceived cost nearly enough to counter the tacit "I will beat the odds mentality" to which criminals and lottery-ticket buyers cling.
In the case of spam, I'm not entirely convinced that any of the three criteria are met, but cranking up the third is certainly not "a solution" as the parent indicated.
Here's an even more effective method: almost all spam contains one of the letters {a, e, i, o, u}. Simply write a grep filter to reject all such messages!
One of the great features of email is immediacy. I want that receipt for my airplane ticket right now, not in a few {minutes, hours, whatever}. If a colleague in Europe or Asia sends me a message and it gets delayed a few {minutes, hours, whatever} it can easily cost a day's delay in our correspondence. I'll tolerate none of that.
We have no way of knowing how many legitimate delivery failures are caused by greylisting. That's because, as the parent points out, messages are rejected a priori and there's no quarantine to check. If you reject and for whatever reason it is not retransmitted, your mail is lost. Maybe this "shouldn't" happen but it does, and it happens often enough that it is not entirely obvious that its false positive rate is less than that of a spam filter.
It is also trivial for a spammer to defeat greylisting. Perhaps they don't at this time, but at any moment they could flip a switch and render your approach useless. Contrary to popular belief, state-of-the-art spam filters aren't so easily defeated.
Blacklisting doesn't suffer from the immediacy problem of greylisting, but it shares the problem of an unknown false positive rate, and mediocre false negative rate.
The volume of spam is definitely up, and most of it is pump and dumps from a very few distinct sources. In December, about 20% of the 30,000 spams I received were for one particular stock.
But it is wrong to say that this new spam requires radical new filtering techniques. That's what the spam solution vendors (whose press releases drive these/. articles) want you to believe so you'll buy their products. In general, word salads, obfuscated words and image spam do not defeat state-of-the-art statistical filters.
These results show that filters achieve about the same results on 2006 spam as on 2004 spam, and those results are pretty good. Ongoing tests show that the effectiveness of filters is unchanged for 2007. In general, the volume of spam has increased, and spammers have tried various methods of defeating spam filters. But their efforts have not been particularly successful against statistical filters.
I just want to add: you can only shoot one bb at the photon; doing so destroys it. So you have to decide which particular bit of information you want to retrieve, and retrieving that bit renders the rest permanently irretrievable.
Quantum computing is very fun and mind-bending, and would facilitate lots of computation that we currently think of as "impossible." Being able to do encodings such as those (mis)described in the article would be one consequence.
After some effort, I found the actual article. The popular press account was bad, even for the popular press, failing to give the title of the paper and giving the author's name only parenthetically.
The article contains at least one claim to "significance at the 5% level" but as far as I can see it is a working paper, not (yet) published in a refereed venue. The author appears to have other credible publications relating to the effect of windfalls on people.
You're saying the position is valid. That's not what the thread is about. My point is that an organization has both the right and the moral obligation to decline to sanction that which it considers to be invalid and harmful.
If a lawyer were to pronounce that it was OK to rob convenience stores, do you think the law society might move to withdraw their seal of approval? What about a physician who advocated -- on a medical TV show -- leeches & bleeding?
It is not a censorship issue for an organization to decline to lend its approval to unsupportable opinions. Since meteorology aims to be a scientific discipline, it seems reasonable to me that it should avoid blessing the expression of opinions contrary to evidence as if they were fact.
Crackpots and demagogues may say what they wish but their freedom to say so is not accompanied by the right to demand seals of approval for their behaviour.
I can't help but think that abandoning the field of CS as a career because there are some 'dweebs' associated with it is stunningly superficial.
No doubt the women who have this comment brought to their attention will be alerted to the superficiality of their feelings, and therefore adjust them to join a culture in which the judgmental attitutes espoused by the author flourish.
For whatever reasons, the software culture has evolved in a way that many women (and many men, but not in as large numbers as women) find it unattractive. That is not so say that they find computer software unappealing; rather, they never get close enough to find out. They see nerds lacking hygiene and basic social skills congregating to learn the arcane details of some system -- or combative video game -- with little thought to how the system might be used to do something useful, artistic or social. Their first exposure is, likely as not, through members of that demographic group.
Women in general tend to be unimpressed by those whose ego exeeds their abilities -- a personality that is all-too-often rewarded in this information economy.
Lots of people (men) want to attract women to computing but have no idea how. Bill Gates came (here) to Waterloo to try to attract non-hard-core-nerds to study CS. My daughter was very keen to see him but after he demo-ed his XBOX 360 and a fingerprint-reading PDA and a Napoleon Dynamite video she came away saying "what a dweeb!" She may end up studying CS, but if she does, it'll be in spite of efforts like that. And two year of high school CS in which she was top of her class, but learned nothing. More likely she'll study math or physics or something that she feels is more challenging and useful, and less associated with dweebs.
The problem with content based filtering is it either increases the amount of wading due to quality control needs or decreases the amount of wading at the expense of lost messages.
There's no evidence that the statement above is true. A user who has to wade through a mixture of spam and non-spam will overlook some of the non-spam. The question is whether the human or the machine will overlook more. A subsidiary question is, once overlooked, how likely is the message to be retrieved using some subsidiary mechanism (second look, scanning the quarantine, whatever). There *is* evidence that content filters are better than humans at the initial separation of good email and spam, *and* that separate good and quarantine folders improve performance on the second task.
Here are two content-based filters that work very well: OSBF-Lua and Bogofilter. SpamAssassin's "Bayes filter" works well, too, but you have to configure it a bit differently: http://plg.uwaterloo.ca/~gvcormac/spamassassin
I wouldn't bash content filtering if I thought it worked
You go a lot further than saying I don't think it works. You pronounce from great heights that it cannot possibly work. Such dismissive statements are without merit.
Content based filtering is NOT working and will NEVER work!
I don't usually respond to ACs, but this particular belief is common enough that I feel I should say a few words. The overall goal of spam abatement is to enhance the probability that legitimate email will be delivered in a timely and efficient manner to its intended recipient. Content-based filtering is widely deployed in this context and it is fairly effective for its intended purpose. Demonstrably more effective, and less intrusive, than forcing the recipient to wade through spam and triage mail manually. And demonstrably more effective, and less intrusive, than refusing or challenging unfamiliar email as a matter of course.
To the extent that we measure these aspects -- risk of non-delivery, delay, intrusiveness of solutions -- the world will be better off. Unsubstantiated dismissal of a particular approach -- especially one for which there is extensive evidence of its efficacy -- is unhelpful.
The Bayesian analysis in spam filters only works on text. Spammers realized that they could get around it by filling the text portion of the message with some random passage from a Project Gutenberg file, thus making it seem innocuous, and then putting the real advertisement in a GIF or PNG file that would be displayed by HTML-capable mail readers. Bayesian analysis can still work, but only in combination with OCR software.
Bayesian filters (and other statistical filters colloqually known as Bayesian) can work on any features at all; not necessarily text. In particular they can use the markup in the header of the message, the message encoding, and so on. Some of the best-performing filters don't use 'text' at all and simply treat the entire message, images and all, as a bit string; for example, compression-based filters. Another well performing filter, OSBF-Lua, uses orthogonal sparse binomial bigrams rather than individual tokens.
Recent standardized testing shows that these methods work just fine on image spam, without any OCR component.
While IJCAI is a prestigious conference, and the results may be sound, the claims as to the applicability to spam filtering are bogus. The paraphrasal of how state-of-the art filters work is wrong, and there's no evidence that better word associations translate to better spam filter accuracy. None at all.
Should the authors wish to show applicability to spam filtering, they should do so using the TREC Spam Track methodology and datasets. http://trec.nist.gov/data/spam.html
The call for participation in TREC 2007 is currently open: http://trec.nist.gov/call07.html Nothing at all prevents a TREC participant from submitting a filter that includes a copy of Wikipedia, if they feel it would help.
Give a drug, which you think makes people sick, to 1 person at random out of 100, and a placebo to the rest.
I'm unaware of any valid experimental design that uses a sample size of 1. A more reasonable design would be to treat 50 and give a placebo to 50 and see if the proportions showing side effects in the two groups differ by more than what could reasonably be attributed to chance.
As far as random screening is concerned, you must consider the positive and negative predictive values of the tests. A very good test might have, for example, a 1% false positive rate and a 2% false negative rate. It is commonly assumed -- falsely -- that a test with 1% false positive rate has 99% predictive value; that the subject is 99% likely to have whatever is being tested for.
From the false positive and false negative rates you have to compute the positive (and negative) predictive value -- that is, the probability that somebody who tests positive (or negative) really has (or does not have) what the test shows. To compute positive predictive value you need to know the prevalance in the population being tested. Suppose the prevalance is 1 in 1000 and you test at random. That means that for every true positive you'll get ten false positives. That is, the positive predictive value is 9%. A far cry from 99%!
Now to compute false positive and negative rates of the order of 1%, you need sample sizes of at least several hundred -- probably thousands. I don't believe there is any physiological test of truthfulness that has shown anything even resembling a statistically significant result, which is why I take great exception to your statement:
even if the commonly-used polygraph is a sham, correct use of science and statistics can be used to devise a better method, and hopefully things will continue to progress in that direction.
The controlled experiments for polygraphs have shown between 40% and 70% false positive and false negative rates which, for the sample sizes used, are indistinguishable from chance.
The effectiveness of the proposed method -- using a polygraph to detect a reaction to the murder weapon -- is unsupported by any evidence. And the statistical argument is bogus.
If 1 in 100 people shows a response, it definitely does *not* mean that person is associated with the crime (p Inappropriate use of statistics -- such as that in this thread -- is dangerous. http://en.wikipedia.org/wiki/Prosecutor's_fallacy
E. Coli is an essential part of our (and animals') digestive process. Most strains of E. Coli are harmless to humans, but some, like O157:H7 are extremely virulent in humans but harmless to the animals that carry them.
It makes sense to spend our efforts trying to eradicate these strains in the domestic and wild animal populations. Otherwise, we run the risk of every farmer's field and every outdoor trail becoming a serious health hazard.
So while the epidemiologic effort to trace the source of the human outbreak is impressive, I think research into controlling it in animals is even more important.
The assertion that a program must make as many mistakes as the human that programmed it is preposterous. I daresay I can write a program that computes a million sums and it'll get more of them right than your average human.
Content-based spam filters can be much more accurate than humans. In particular, they can have lower false positive rates. That is, a good spam filter is less likely to discard good email than a human is to overlook good email in a sea of spam.
I'm not exactly sure how the article supports the title "It's not worth worrying about spam." Does this mean you freely distribute your email address, and you simply sort through all your messages by hand, and you've never overlooked a good email, and you have some way of knowing whether or not this is the case?
If you want to test your own ability to separate spam from good email, visit www.spamorham.org
I suggest that you use a statistical spam filter instead. Training its (few) errors is all-in-all less work and more effective than composing ad hoc rules. Even if you use Spamassassin, just turn the Bayes way up and forget the ad hoc rules. But there are better statistical filters. OSBF-Lua is the best (at least the best available) and Bogofilter is also very good, and more mature.
Except PHYA is just the lure to get you to the "broker" site. Then they bait-n-switch you to another stock. By the time the exchange notices the scammers are long gone.
Slashdot Mirror
Indeed every mail provider should have such an interface: a trivial way to report filtering mistakes. But you over-estimate the value of everybody else's spam reporting. A filter based only on your own reporting can have a vanishingly small number of false positives, and a small number of false negatives. So small that the total amount of reporting you have to do is no more than for Gmail.
But many appliance manufacturers promote the scenario in which the user is not prepared to offer any feedback to the filter. It is much harder to achieve reasonable error rates in this mode of operation.
Bottom line: Gmail's filter is pretty good, but not better than the personal spam filters I've tested. I have yet to see a "hands-free" solution that is as good as one that uses feedback. The amount of feedback required is trivial.
If you want to stop crime, the penalty should be,
and perceived to be:
- certain
- immediate
- more costly than the benefit of the crime
"Law and order" advocates generally advocate
draconian punishments, but there is no evidence
that they help, beyond counterbalancing the
benefit of the crime. Increased detection speed
and likelihood are far more effective.
You might think that draconian punishments increase
the expected cost, even with haphazard and delayed
detection, but they don't increase the perceived
cost nearly enough to counter the tacit "I will
beat the odds mentality" to which criminals and
lottery-ticket buyers cling.
In the case of spam, I'm not entirely convinced
that any of the three criteria are met, but
cranking up the third is certainly not "a solution"
as the parent indicated.
Here's an even more effective method: almost all spam contains one of the letters {a, e, i, o, u}. Simply write a grep filter to reject all such messages!
We have no way of knowing how many legitimate delivery failures are caused by greylisting. That's because, as the parent points out, messages are rejected a priori and there's no quarantine to check. If you reject and for whatever reason it is not retransmitted, your mail is lost. Maybe this "shouldn't" happen but it does, and it happens often enough that it is not entirely obvious that its false positive rate is less than that of a spam filter.
It is also trivial for a spammer to defeat greylisting. Perhaps they don't at this time, but at any moment they could flip a switch and render your approach useless. Contrary to popular belief, state-of-the-art spam filters aren't so easily defeated.
Blacklisting doesn't suffer from the immediacy problem of greylisting, but it shares the problem of an unknown false positive rate, and mediocre false negative rate.
The volume of spam is definitely up, and most of it is pump and dumps from a very few distinct sources. In December, about 20% of the 30,000 spams I received were for one particular stock.
1 4241
/. articles) want you to believe so you'll buy their products. In general, word salads, obfuscated words and image spam do not defeat state-of-the-art statistical filters.
http://it.slashdot.org/article.pl?sid=06/12/21/23
But it is wrong to say that this new spam requires radical new filtering techniques. That's what the spam solution vendors (whose press releases drive these
See, for example, the recent TREC tests: http://plg.uwaterloo.ca/~gvcormac/trecspamtrack06
These results show that filters achieve about the same results on 2006 spam as on 2004 spam, and those results are pretty good. Ongoing tests show that the effectiveness of filters is unchanged for 2007. In general, the volume of spam has increased, and spammers have tried various methods of defeating spam filters. But their efforts have not been particularly successful against statistical filters.
Quantum computing is very fun and mind-bending, and would facilitate lots of computation that we currently think of as "impossible." Being able to do encodings such as those (mis)described in the article would be one consequence.
After some effort, I found the actual article. The popular press account was bad, even for the popular press, failing to give the title of the paper and giving the author's name only parenthetically.
In any event, here is the article: http://ideas.repec.org/p/wrk/warwec/785.html
The article contains at least one claim to "significance at the 5% level" but as far as I can see it is a working paper, not (yet) published in a refereed venue. The author appears to have other credible publications relating to the effect of windfalls on people.
Not only that, don't fly near US air space in case you are grounded due to an emergency.
a lth&res=9B0CE1DE1531F933A25752C0A962958260
http://query.nytimes.com/gst/fullpage.html?sec=he
You're saying the position is valid. That's not what the thread is about. My point is that an organization has both the right and the moral obligation to decline to sanction that which it considers to be invalid and harmful.
If a lawyer were to pronounce that it was OK to rob convenience stores, do you think the law society might move to withdraw their seal of approval? What about a physician who advocated -- on a medical TV show -- leeches & bleeding?
It is not a censorship issue for an organization to decline to lend its approval to unsupportable opinions. Since meteorology aims to be a scientific discipline, it seems reasonable to me that it should avoid blessing the expression of opinions contrary to evidence as if they were fact.
Crackpots and demagogues may say what they wish but their freedom to say so is not accompanied by the right to demand seals of approval for their behaviour.
I can't help but think that abandoning the field of CS as a career because there are some 'dweebs' associated with it is stunningly superficial.
No doubt the women who have this comment brought to their attention will be alerted to the superficiality of their feelings, and therefore adjust them to join a culture in which the judgmental attitutes espoused by the author flourish.
For whatever reasons, the software culture has evolved in a way that many women (and many men, but not in as large numbers as women) find it unattractive. That is not so say that they find computer software unappealing; rather, they never get close enough to find out. They see nerds lacking hygiene and basic social skills congregating to learn the arcane details of some system -- or combative video game -- with little thought to how the system might be used to do something useful, artistic or social. Their first exposure is, likely as not, through members of that demographic group.
Women in general tend to be unimpressed by those whose ego exeeds their abilities -- a personality that is all-too-often rewarded in this information economy.
Lots of people (men) want to attract women to computing but have no idea how. Bill Gates came (here) to Waterloo to try to attract non-hard-core-nerds to study CS. My daughter was very keen to see him but after he demo-ed his XBOX 360 and a fingerprint-reading PDA and a Napoleon Dynamite video she came away saying "what a dweeb!" She may end up studying CS, but if she does, it'll be in spite of efforts like that. And two year of high school CS in which she was top of her class, but learned nothing. More likely she'll study math or physics or something that she feels is more challenging and useful, and less associated with dweebs.
We Canadians take "American" to mean a citizen of the USA; not of Canada, Mexico, Brazil or Argentina.
There's no evidence that the statement above is true. A user who has to wade through a mixture of spam and non-spam will overlook some of the non-spam. The question is whether the human or the machine will overlook more. A subsidiary question is, once overlooked, how likely is the message to be retrieved using some subsidiary mechanism (second look, scanning the quarantine, whatever). There *is* evidence that content filters are better than humans at the initial separation of good email and spam, *and* that separate good and quarantine folders improve performance on the second task.
Here are two content-based filters that work very well: OSBF-Lua and Bogofilter. SpamAssassin's "Bayes filter" works well, too, but you have to configure it a bit differently: http://plg.uwaterloo.ca/~gvcormac/spamassassin
You go a lot further than saying I don't think it works. You pronounce from great heights that it cannot possibly work. Such dismissive statements are without merit.
I don't usually respond to ACs, but this particular belief is common enough that I feel I should say a few words. The overall goal of spam abatement is to enhance the probability that legitimate email will be delivered in a timely and efficient manner to its intended recipient. Content-based filtering is widely deployed in this context and it is fairly effective for its intended purpose. Demonstrably more effective, and less intrusive, than forcing the recipient to wade through spam and triage mail manually. And demonstrably more effective, and less intrusive, than refusing or challenging unfamiliar email as a matter of course.
To the extent that we measure these aspects -- risk of non-delivery, delay, intrusiveness of solutions -- the world will be better off. Unsubstantiated dismissal of a particular approach -- especially one for which there is extensive evidence of its efficacy -- is unhelpful.
Bayesian filters (and other statistical filters colloqually known as Bayesian) can work on any features at all; not necessarily text. In particular they can use the markup in the header of the message, the message encoding, and so on. Some of the best-performing filters don't use 'text' at all and simply treat the entire message, images and all, as a bit string; for example, compression-based filters. Another well performing filter, OSBF-Lua, uses orthogonal sparse binomial bigrams rather than individual tokens.
Recent standardized testing shows that these methods work just fine on image spam, without any OCR component.
http://www.ijcai.org/papers07/Papers/IJCAI07-259.p df
While IJCAI is a prestigious conference, and the results may be sound, the claims as to the applicability to spam filtering are bogus. The paraphrasal of how state-of-the art filters work is wrong, and there's no evidence that better word associations translate to better spam filter accuracy. None at all.
Should the authors wish to show applicability to spam filtering, they should do so using the TREC Spam Track methodology and datasets. http://trec.nist.gov/data/spam.html
The call for participation in TREC 2007 is currently open: http://trec.nist.gov/call07.html Nothing at all prevents a TREC participant from submitting a filter that includes a copy of Wikipedia, if they feel it would help.
As far as random screening is concerned, you must consider the positive and negative predictive values of the tests. A very good test might have, for example, a 1% false positive rate and a 2% false negative rate. It is commonly assumed -- falsely -- that a test with 1% false positive rate has 99% predictive value; that the subject is 99% likely to have whatever is being tested for.
From the false positive and false negative rates you have to compute the positive (and negative) predictive value -- that is, the probability that somebody who tests positive (or negative) really has (or does not have) what the test shows. To compute positive predictive value you need to know the prevalance in the population being tested. Suppose the prevalance is 1 in 1000 and you test at random. That means that for every true positive you'll get ten false positives. That is, the positive predictive value is 9%. A far cry from 99%!
Now to compute false positive and negative rates of the order of 1%, you need sample sizes of at least several hundred -- probably thousands. I don't believe there is any physiological test of truthfulness that has shown anything even resembling a statistically significant result, which is why I take great exception to your statement:
The controlled experiments for polygraphs have shown between 40% and 70% false positive and false negative rates which, for the sample sizes used, are indistinguishable from chance.
If 1 in 100 people shows a response, it definitely does *not* mean that person is associated with the crime (p
Inappropriate use of statistics -- such as that in this thread -- is dangerous. http://en.wikipedia.org/wiki/Prosecutor's_fallacy
It makes sense to spend our efforts trying to eradicate these strains in the domestic and wild animal populations. Otherwise, we run the risk of every farmer's field and every outdoor trail becoming a serious health hazard.
So while the epidemiologic effort to trace the source of the human outbreak is impressive, I think research into controlling it in animals is even more important.
PHYA ad seems to have been withdrawn. At first I thought maybe they'd been shut down but apparently that entry just became too expensive.
R GJ|THRI.PK|WBRS|WBRS
Here are some others to try:
ARSS|CNPM|CVNI.PK|DTGP|LITL|PHYA|PKGH|PMHD|PPTL|P
Parent does not understand grandparent. The Google ad points to a stock market manipulator, not PHYA.
Content-based spam filters can be much more accurate than humans. In particular, they can have lower false positive rates. That is, a good spam filter is less likely to discard good email than a human is to overlook good email in a sea of spam.
I'm not exactly sure how the article supports the title "It's not worth worrying about spam." Does this mean you freely distribute your email address, and you simply sort through all your messages by hand, and you've never overlooked a good email, and you have some way of knowing whether or not this is the case?
If you want to test your own ability to separate spam from good email, visit www.spamorham.org
I suggest that you use a statistical spam filter instead. Training its (few) errors is all-in-all less work and more effective than composing ad hoc rules. Even if you use Spamassassin, just turn the Bayes way up and forget the ad hoc rules. But there are better statistical filters. OSBF-Lua is the best (at least the best available) and Bogofilter is also very good, and more mature.
Except PHYA is just the lure to get you to the "broker" site. Then they bait-n-switch you to another stock. By the time the exchange notices the scammers are long gone.