OK, well, Hotmail blows. But that just means they are doing it wrong. (Plus, the conspiracy theorist in me says they don't want to filter spam.)
They aren't just doing pattern matching; it's more sophisticated than that. It is also adaptive. As Paul Graham said, you can defeat spammers this way because they rely on their message. Email clients can do whitelisting techniques to reduce or eliminate false negatives as well as other things. This can all be done behind the scenes, with user interaction limited to the initial training of spam and the discovery of false positives. We have the technology! No, filtering hasn't been defeated nearly as far as I can see.
It works for my mother and all of her employees perfectly. A few questions: How are you training your Thunderbird install? What do you mean "contaminated?" And why the hell would you *delete* filtered spam immediately? The idea is to save that spam for a while (30 days is good) in a "Trash" or "Recycle Bin"(patent pending) just in case one gets through. Someone notifies you that you aren't responding, you dig it up, classify it, and your filter gets better. But if you spend long enough with a spam filter, filtering it correctly, you will generally not get false negatives.
There is a price to smart filtering: you have to spend time with it to train it correctly. If you train it wrong, you've got a huge problem on your hands. I've said it elsewhere in the comments: look at CRM114 to see how good this kind of filtering has become, and how quickly you don't have to worry about it. But I personally have never lost an important, critical email with Thunderbird or Gmail. Neither have I heard a single complaint about spam from any Gmail user.
I do however, agree with you that ISPs should not be filtering your spam for you. That just gets annoying. But rejecting spam from IP addresses is an idea that can only go so far: like you said, spammers have huge swaths of IP addresses, sometimes ones that are used by legit emailers. IPv6 is coming, which means even more IP addresses for you to block. Personally, I think client-side filtering is quickly becoming the superior spam solution - look at that "SPAM solution checklist" that someone else posted.
Personally, it's been a long time since I worried about spam.
My Gmail account has a success rate of about 2/1000 or 99.8% success rate. My Thunderbird email has a similar success rate. Speak for yourself, buddy, statistical filtering works.
Excellent analogy, but that's all there is. It might be inspiring, but this time the idea wasn't originally inspired by biology. These methods of filtering spam have been around for a long time.
In any case, the basic idea is simple: use a corpus of examples separated into classes to create an algorithm to decide if a new example is in a certain category. There are million AI techniques to do this. What differs in each case are the details of what each part means.
The immune system analogy is flawed in its details anyway. For example, in the human body, antibodies are created more like a genetic algorithm: there are a few families of them and they recombine randomly and float around the body. (Those harmful to the body are never let out.) Those that find a matching host invader protien are then made to reproduce. Should we implement a similar strategy for spam? Probably not, or at least my intuition says that this method does not work as well with spam as some of the very very successful strategies that we have now, especially most of these converge very quickly. GA-style seems like it would take a long time.
There are a million other biological ideas we could borrow, and other biological ideas we could borrow in radically different ways, but we don't because they don't work. Those ideas that do work may have been inspired by biological phenomena, but other than that they do little better than provide a good analogy. In this case, they aren't doing anything different and it is only considered interesting because they thought of a good analogy for it. Nothing works because it is based on biological phenomena. I realize that much of AI research is well grounded in mathematical theory (heck, my master's advisor does stuff with COLT and the like), but many students, Slashdotters, and a few researchers still have a romantic kind of view of "intelligence" and "living computers" or whatever. So basically my comment was directed more at Slashdot than the research group. Hell, I didn't RTFA, so they could be doing Serious Research.
The flip side to what you are saying of course is that accepting a work just because it is biologically inspired is also inappropriate and counter-productive to science.
I'm seriously sick of people abusing biological methodolgies. People seem very attracted to ideas simply because they are grounded in "how nature works" and ignore the mathematical benefits or weaknesses. Now this idea pretty much just sounds like statistical rules based on a corpus - pretty much how every successful solution out there now works. This solution simply prunes rules that aren't being used, but there are better ways to get a smaller spam detection database. Have you seen the stuff the CRM114 people are doing? This is nothing new.
Read your Russell and Norvig, people. Airplane research didn't get off the ground (ugh) until we stopped trying to mimic birds and study physical principles of flight.
Excellent idea! Let's round up all the people who refuse to stop practicing their religion. That'll teach those religious bastards to discriminate against people based on their religion!
If your idea works, call me when you need help figuring out how to stop crazed idealists who have decided that killing for secular belief X is the moral way of doing things. You know, like Stalin.
Of course he had all the right info, because it was a good prank. And absolutely, you have to treat it as a normal call - it can often be hilarious for both parties - but don't go around thinking there's a black supremacist out there who hates his mouse cursor cause it's white.
This was at least a year ago, and my Internet provider is from an entirely different country now. I think they may have even changed the hardware that caused this behaviour.
I had a very similar experience with Comcast. This time, I could not connect to the internet at all if the modem hooked up through my router. Now, I know my router is working, so I called up tech support, but I actually listened to them and plugged my computer directly into the cable modem. Bam, it works. So, they say, the problem is clearly something wrong with the router. I protest and say that, no, the router is not causing any difficulties, one of your gateways is down. They swore that nothing was wrong with any of their systems. So I called my router's manufacturer (Linksys).
After trying a million tests, they suggested that I set the router to copy the MAC Address from my computer.
Bam, it works.
It must have that they assigned me an IP number (and therefore gateway) based on my MAC Address, and the one it gave my router was not working properly. I called the Comcast representative back, to notify them that there was indeed a problem, and that their tech support was giving me non-answers. I'm sure it didn't help, but it felt better.
I worked in tech support doing internal support for a chain of gas stations. They had Windows 95 and Microsoft Exchange set up to send the central office its sales figures everyday. One day an emmployee calls in and says "Help! My computer is frozen! What should I do?" This happens a lot, because the modems we used caused conflicts with the mouse somehow, and the program we used was buggy, and about a million other reasons that had me questioning the capability of the Sysadmin. So I asked, "OK, what do you see on your screen?"
She replied, "It says 'It is now safe to turn off your computer.'"
During the previous contest, the U of C organizers wanted to make sure that the winner used solid programming techniques than spit out a 3-day hack fest. They had contestants make a program - and then after they submitted their program, the rules changed and they had to make additions to their program. Pretty slick if you ask me.
A patent lawsuit may be the only option a small guy with an idea stolen by a big corp.
This never happens.
You know the story: inventor invents "brilliant" new software (for example, let's say using a "hand held device" to "scroll through" a "list of currently running programs"), and wants to market it. Another company decides, "great idea," nicks it from him, and he threatens to sue. "Ah," the big companies say, "we have 5 patents that you cannot develop your software without!" Mr. Small Inventor is screwed.
Secondly, OSS software generally either apes current computer ideas or, when it is innovative, it usually takes large corporations years to catch up (tabbed browsing, anyone?) Most major research comes from unviersities or is freely available from the corporations that do it (e.g. Microsoft Research).
So: 1) Software patents don't encourage innovation 2) Nearly all (I'd wager 99%) of software patents are trivial given the problem that they are meant to solve 3) Software patents only help larger corporations 4) The OSS Market can thrive on doing ideas better and implementing them faster. (CATB theory)
What's the new model then? Just chuck 'em. We already have protection for software that fits it: copyright. Software does not take the large amount of time and capital (it takes zero capital really) to develop that physical processes do: no factory to buy, and sometimes there are no workers to hire. Not only that, it is free to reproduce. Software companies generally do not need and do not deserve the monopoly over their "invention."
So active cookies totally won't work for certain large sites like yahoo or google that have services like akadns which change the resolved IP addresses quite often?
Yes, the headline is a sham (it's unauthorized dling only), but still those record/movie industry people are really really picky:
"Compared to some European countries, Spain has some way to go in enforcement, but the new intellectual property law is a definite step forward, placing obligations for instance on ISPs to provide information. Hopefully, it will help us to get some injunctions."
This pretty much hands them all they need to protect their copyright and they act like they are still hurting??
The real solution is to make all of ebay sealed second-place bidding style. That way you can bid early and it's just as good as a snipe. The extreme length of time and lack of real-time bidding makes English proxy bidding an awful solution.
MOD PARENT UP. This is a wonderful economic/game theory analysis of exactly what everyone is talking about. Mod parent to 6 or even better, delete everything else and just keep this link.
The problem with ebay is that the auctions are too long and not in real time. These are what make actual auctions work. Let's say we have an item I for which I am willing to pay $350 but in reality I will go to $400. But I would much rather pay $10 or even $50 for the item. Now, the item is currently bidding for $15. "Aha!" I think, "I can get this for a steal if I bid only $20." But then I think, "Now if someone else starts bidding, and their max is below mine, I am going to have to pay more for this item. So I should keep my max secret until the last possible second." So you basically have everyone sitting around and not bidding because they don't want to push the bid up anymore than it needs to go. The result is sniping, and ebay's current method of auction is just not working otherwise.
Slashdot Mirror
OK, well, Hotmail blows. But that just means they are doing it wrong. (Plus, the conspiracy theorist in me says they don't want to filter spam.)
They aren't just doing pattern matching; it's more sophisticated than that. It is also adaptive. As Paul Graham said, you can defeat spammers this way because they rely on their message. Email clients can do whitelisting techniques to reduce or eliminate false negatives as well as other things. This can all be done behind the scenes, with user interaction limited to the initial training of spam and the discovery of false positives. We have the technology! No, filtering hasn't been defeated nearly as far as I can see.
It works for my mother and all of her employees perfectly. A few questions: How are you training your Thunderbird install? What do you mean "contaminated?" And why the hell would you *delete* filtered spam immediately? The idea is to save that spam for a while (30 days is good) in a "Trash" or "Recycle Bin"(patent pending) just in case one gets through. Someone notifies you that you aren't responding, you dig it up, classify it, and your filter gets better. But if you spend long enough with a spam filter, filtering it correctly, you will generally not get false negatives.
There is a price to smart filtering: you have to spend time with it to train it correctly. If you train it wrong, you've got a huge problem on your hands. I've said it elsewhere in the comments: look at CRM114 to see how good this kind of filtering has become, and how quickly you don't have to worry about it. But I personally have never lost an important, critical email with Thunderbird or Gmail. Neither have I heard a single complaint about spam from any Gmail user.
I do however, agree with you that ISPs should not be filtering your spam for you. That just gets annoying. But rejecting spam from IP addresses is an idea that can only go so far: like you said, spammers have huge swaths of IP addresses, sometimes ones that are used by legit emailers. IPv6 is coming, which means even more IP addresses for you to block. Personally, I think client-side filtering is quickly becoming the superior spam solution - look at that "SPAM solution checklist" that someone else posted.
Personally, it's been a long time since I worried about spam.
Forrest Gump did this first.
My Gmail account has a success rate of about 2/1000 or 99.8% success rate. My Thunderbird email has a similar success rate. Speak for yourself, buddy, statistical filtering works.
Excellent analogy, but that's all there is. It might be inspiring, but this time the idea wasn't originally inspired by biology. These methods of filtering spam have been around for a long time.
In any case, the basic idea is simple: use a corpus of examples separated into classes to create an algorithm to decide if a new example is in a certain category. There are million AI techniques to do this. What differs in each case are the details of what each part means.
The immune system analogy is flawed in its details anyway. For example, in the human body, antibodies are created more like a genetic algorithm: there are a few families of them and they recombine randomly and float around the body. (Those harmful to the body are never let out.) Those that find a matching host invader protien are then made to reproduce. Should we implement a similar strategy for spam? Probably not, or at least my intuition says that this method does not work as well with spam as some of the very very successful strategies that we have now, especially most of these converge very quickly. GA-style seems like it would take a long time.
There are a million other biological ideas we could borrow, and other biological ideas we could borrow in radically different ways, but we don't because they don't work. Those ideas that do work may have been inspired by biological phenomena, but other than that they do little better than provide a good analogy. In this case, they aren't doing anything different and it is only considered interesting because they thought of a good analogy for it. Nothing works because it is based on biological phenomena. I realize that much of AI research is well grounded in mathematical theory (heck, my master's advisor does stuff with COLT and the like), but many students, Slashdotters, and a few researchers still have a romantic kind of view of "intelligence" and "living computers" or whatever. So basically my comment was directed more at Slashdot than the research group. Hell, I didn't RTFA, so they could be doing Serious Research.
The flip side to what you are saying of course is that accepting a work just because it is biologically inspired is also inappropriate and counter-productive to science.
I'm seriously sick of people abusing biological methodolgies. People seem very attracted to ideas simply because they are grounded in "how nature works" and ignore the mathematical benefits or weaknesses. Now this idea pretty much just sounds like statistical rules based on a corpus - pretty much how every successful solution out there now works. This solution simply prunes rules that aren't being used, but there are better ways to get a smaller spam detection database. Have you seen the stuff the CRM114 people are doing? This is nothing new.
Read your Russell and Norvig, people. Airplane research didn't get off the ground (ugh) until we stopped trying to mimic birds and study physical principles of flight.
Excellent idea! Let's round up all the people who refuse to stop practicing their religion. That'll teach those religious bastards to discriminate against people based on their religion!
If your idea works, call me when you need help figuring out how to stop crazed idealists who have decided that killing for secular belief X is the moral way of doing things. You know, like Stalin.
Of course he had all the right info, because it was a good prank. And absolutely, you have to treat it as a normal call - it can often be hilarious for both parties - but don't go around thinking there's a black supremacist out there who hates his mouse cursor cause it's white.
This was at least a year ago, and my Internet provider is from an entirely different country now. I think they may have even changed the hardware that caused this behaviour.
At least now Microsoft is smart amd don't name their operating system after a specific year anymore.
Sorry, but you got pranked. This was obviously a joke call. Maybe you can find it online somewhere.
I had a very similar experience with Comcast. This time, I could not connect to the internet at all if the modem hooked up through my router. Now, I know my router is working, so I called up tech support, but I actually listened to them and plugged my computer directly into the cable modem. Bam, it works. So, they say, the problem is clearly something wrong with the router. I protest and say that, no, the router is not causing any difficulties, one of your gateways is down. They swore that nothing was wrong with any of their systems. So I called my router's manufacturer (Linksys).
After trying a million tests, they suggested that I set the router to copy the MAC Address from my computer.
Bam, it works.
It must have that they assigned me an IP number (and therefore gateway) based on my MAC Address, and the one it gave my router was not working properly. I called the Comcast representative back, to notify them that there was indeed a problem, and that their tech support was giving me non-answers. I'm sure it didn't help, but it felt better.
I worked in tech support doing internal support for a chain of gas stations. They had Windows 95 and Microsoft Exchange set up to send the central office its sales figures everyday. One day an emmployee calls in and says "Help! My computer is frozen! What should I do?" This happens a lot, because the modems we used caused conflicts with the mouse somehow, and the program we used was buggy, and about a million other reasons that had me questioning the capability of the Sysadmin. So I asked, "OK, what do you see on your screen?"
She replied, "It says 'It is now safe to turn off your computer.'"
During the previous contest, the U of C organizers wanted to make sure that the winner used solid programming techniques than spit out a 3-day hack fest. They had contestants make a program - and then after they submitted their program, the rules changed and they had to make additions to their program. Pretty slick if you ask me.
This never happens.
You know the story: inventor invents "brilliant" new software (for example, let's say using a "hand held device" to "scroll through" a "list of currently running programs"), and wants to market it. Another company decides, "great idea," nicks it from him, and he threatens to sue. "Ah," the big companies say, "we have 5 patents that you cannot develop your software without!" Mr. Small Inventor is screwed.
Secondly, OSS software generally either apes current computer ideas or, when it is innovative, it usually takes large corporations years to catch up (tabbed browsing, anyone?) Most major research comes from unviersities or is freely available from the corporations that do it (e.g. Microsoft Research).
So:
1) Software patents don't encourage innovation
2) Nearly all (I'd wager 99%) of software patents are trivial given the problem that they are meant to solve
3) Software patents only help larger corporations
4) The OSS Market can thrive on doing ideas better and implementing them faster. (CATB theory)
What's the new model then? Just chuck 'em. We already have protection for software that fits it: copyright. Software does not take the large amount of time and capital (it takes zero capital really) to develop that physical processes do: no factory to buy, and sometimes there are no workers to hire. Not only that, it is free to reproduce. Software companies generally do not need and do not deserve the monopoly over their "invention."
It won't work - or at least, FOSS won't work, because you can't enforce the GPL that way.
The Matrix has you.
So active cookies totally won't work for certain large sites like yahoo or google that have services like akadns which change the resolved IP addresses quite often?
Yes, the headline is a sham (it's unauthorized dling only), but still those record/movie industry people are really really picky:
"Compared to some European countries, Spain has some way to go in enforcement, but the new intellectual property law is a definite step forward, placing obligations for instance on ISPs to provide information. Hopefully, it will help us to get some injunctions."
This pretty much hands them all they need to protect their copyright and they act like they are still hurting??
-1, Atrocious grammar, misuse of latin abbreviation
The real solution is to make all of ebay sealed second-place bidding style. That way you can bid early and it's just as good as a snipe. The extreme length of time and lack of real-time bidding makes English proxy bidding an awful solution.
To be fair, punctuation marks aren't capable of making very good art. (Or at least whoever did the work was too ashamed to add their name to it.)
Where do I sign up not to ever use Open Office again after seeing that ad?
That ad looks like someone from Fark made it.
MOD PARENT UP. This is a wonderful economic/game theory analysis of exactly what everyone is talking about. Mod parent to 6 or even better, delete everything else and just keep this link.
The problem with ebay is that the auctions are too long and not in real time. These are what make actual auctions work. Let's say we have an item I for which I am willing to pay $350 but in reality I will go to $400. But I would much rather pay $10 or even $50 for the item. Now, the item is currently bidding for $15. "Aha!" I think, "I can get this for a steal if I bid only $20." But then I think, "Now if someone else starts bidding, and their max is below mine, I am going to have to pay more for this item. So I should keep my max secret until the last possible second." So you basically have everyone sitting around and not bidding because they don't want to push the bid up anymore than it needs to go. The result is sniping, and ebay's current method of auction is just not working otherwise.
Personally, I just Buy It Now.