I don't think I'm underestimating the rancor. I've been a direct recipient of some of that rancor and I know that it's not small. However, I think that the size of the population who experiences that rancor is small. Which is to say that the rancor is highly concentrated in a very vocal minority. By far, the biggest reaction I get from correspondants who see my challenges is this: Wow, that's really cool. Can you set that up for me?
I truly wish there was a way for me to prevent innocents from getting unnecessary challenges. And my response to that is this: Please use SPF. If you're using SPF, I won't challenge you when someone joe-jobs your email. As soon as someone comes out with a DomainKeys filter, I'll use that, too. I'll filter on anything that can deterministically identify a forgery. But that's not today's biggest problem. By far, today, most SPAM comes from non-existant email addresses. 95% by my sampling. Which means that most of the challenges produce the desired result: preventing the spam from being delivered. A small amount (5%) get delivered. Of the ones that get delivered, I would agree that most of them go to joe-jobbed accounts.
All I can say about that is "Sorry". I will do whatever I can (short of turning of C/R) to help alleviate that. So if you have some suggestions, please provide them. My suggestions are as follows:
Figure out how to deterministically identify when you've received a bounce/challenge from an email you sent. TMDA can help with this even if you don't use the C/R features it has. (Turning off C/R is as easy as setting the following variable: ACTION_INCOMING=ok)
Use SPF and/or DomainKeys and/or... whatever is available so that you can help others identify forgeries of your domain.
But you're wrong on a number of your facts. I don't think it's an inaccurate conclusion to state that you don't understand something when many of your facts turn up wrong.
There's a published standard which TMDA complies with to announce that it's challenges are machine generated. You can filter on it or not, but it's the way that all machine generated emails are supposed to be advertised.
Well, from actual practice having corresponded with an incredibly broad spectrum of users, I would say that the average mail user is more capable than you give them credit for. Simply replying to a challenge is not terribly difficult for just about anyone who can use email.
And for what it's worth,
manually mainting white/list black lists is very infrequent and optional
no one has to create new accounts for every service that sends email automatically. You simply create a new email address that goes to the same account. This can be done through a webpage.
You certainly can whitelist your friends manually, but in practice this is an optional activity.
This is not unlike using keyword addresses in TMDA. The difference is that you generate the addresses and you don't have to keep track of them anywhere. The address has a hash embedded in it that's generated with a private key. When TMDA sees a keyword address it attempts to regen the hash and if it matches, the email is allowed in.
You might think that this is a major flaw, but most C/R solutions have already anticipated this and come up with a solution. Here's the solution using TMDA.
Yes it is, and I whole heartedly recommend to anyone doing C/R that they first filter incoming email through SPF. Thus they'll be able to identify at least some of the forgeries. If everyone were using SPF, then forgeries would not be a problem.
I discarded such email address manipulation approaches roughly 15 years ago
My comment to this is that C/R requires some ability to encode information to the user in the challenge that must come back in the response. About the only reliable way to do this is by manipulating the address.
There are other C/R solutions that encode the information in the Subject. But it's possible to change the subject in the reply and remove the information. This is why TMDA encodes the information in the email address.
Ah. I see what you're saying. You're trying to use TMDA for a host of users. Well, frankly, I'm unqualified to answer how well that works as I haven't tried it. I use it only for myself, my wife and one of my children old enough to handle email.
At first glance I could see how it might be difficult to manage multiple accounts. That being said, there are free and commercial email services offerinng TMDA to the public. They seem to have figured out how to do so on a scalable fashion. I would recommend you talk to them about how to do it.
Just because I left my door unlocked does not mean you can enter and take stuff. The same applies to email: you are a spammer irrespective of whether or not I protect my mailbox.
Bad analogy. I haven't come into your house. I just had my door locked when some guy tried to break in. Failing the ability to break into my house, he broke into your unlocked door.
In one sense, you could argue that I caused the break into your house. The fact that I locked my door caused him not to break into my house and consequently he broke into your house. But the criminal here is not me for locking my door. Nor is it you for not locking your door. It's the criminal for breaking in.
My suggestion to you is to lock your door. No offense, but I'm going to reject your suggestion that I unlock mine.
You are entitled to your opinion. I disagree. Just FYI, careful how you defend yourself or your family. Because if I'm a spammer, it takes no effort for me to concocat a scenario which (by your standard) makes you a murderer for defending yourself.
While I wish I were Jason Masteler, I, unfortunately am not.
It is true, however, that you do need your own domain. But the address manipulation is not at the domain level. It's at the username level. E.g. joe@blow.com becomes joe-dated-1102294815.bed09c@blow.com. You need only 1 domain, and you generate the rest from the username.
I don't really think you know how TMDA works. I would suggest you investigate it before you make claims about it's scalability.
Just a thought. Ignore it if it's not useful to you.
Here's another scenario: using agressive spam filters, your "oppurtunity" gets miscategorized as spam, and I never even know that you sent it to me. You conclude that I don't care for the oppurtunity, and that's the end of the story.
At least with C/R, you KNOW that my spam filter has prevented me from receiving your email. With all other spam filters, it filters silently so that NO ONE knows that it's been filtered. If it doesn't filter silently, one of us has to be notified.
If I'm notified of all email coming in, that's functionally equivalent to turning off spam filterinng. If you're notified, that's functionally equivalent to C/R. According to the anti-C/R crowd, the only acceptable thing to do is turn off spam filtering. I hope they practice what they preach.
How can you expect someone's grandmother to use such a system?
Well a surprisingly large number of grandmothers do successfully respond to the C/R that I send. The hard part is if the grandmother wants to use C/R on her email account. That's a much more difficult problem to explain.
if somebody (even a friend) uses them I won't bother unlocking with a response, and I won't use email to contact them again. It's their loss, not mine.
I don't see how this is my (or any other C/R user's) loss. You're the one who sent the email in the first place. Presumably you had some reason for doing so. Whatever that reason was, that's what is lost. If your email wasn't important enough to you to ensure it gets to me, then I don't feel much loss for having ignored it.
And you're a very strange friend, who values a particular email system over your relationship with that person. Good luck with that.
I don't know in what universe it's a useful point to mention that you're removing invalid email address before you send mail to them.
I'm not removing invalid email addresses before I send email to them. I'm removing it after I discover that they're invalid. How exactly does anyone know if an email address is valid before probing to see if it's valid?
For every legitmate message you receive, 24 other people had to look at a spam you sent them.
Wrong again. The spam originated from someone else. My system didn't stop the spam, but you can't call my replying to an email address as a spam. That's like saying that you're a murderer if you defend yourself when a knife bounces off of your shield into someone else. The person responsible is the person who threw the knife, not the person who defended themself.
I'm sorry that someone who joe-jobs your email address causes you problems. And I'm sorry that I don't have enough information to prevent them from abusing my system to annoy you. But remember who is causing the problem: the joe-jobber. The joe-jobber is responsible for sending the spam to you in EXACTLY the same way that he's responsible for causing your email address to get bounces from bad joe-jobbed email.
My best recommendation to you:
publish an SPF record for your domain. I filter incoming email and respect publish SPF records.
figure out a sensible way to reliably determine when a bounce is correctly or incorrectly from you. It's not rocket science. If I can do it, so can you.
Or don't. I don't care how you choose to defend your mailbox. Just stop calling other people "spammer" when you choose to leave your box unprotected.
One more thing. I don't know if this is true for ALL C/R, but TMDA goes out of it's way to announce that the challenge is computer generated.
It sets the "Precedence: bulk" header.
It sets the "Auto-Submitted: auto-replied" header.
In the body of the message, as the first line, it says: "This message was created automatically by mail delivery software (TMDA)."
The confirmation is actually a MIME attachment with the following MIME header: "Content-Description: Confirmation Request".
If you don't want to receive TMDA challenges, that's no problem. You can EASILY filter for any of these things and you'll never receive a TMDA challenge ever.
Wrong. You assume that all email I challenge comes from legitimate addresses that are forged. While you're right that the number of spams that come in from forged addresses is greater than zero, it's no where near 100%.
I used to have a program installed that would delete emails from my pending list that were from invalid adddresses. An invalid address was determined by getting a bounce when it failed delivery. The bounce would contain a reference to the file in my pending list and it was pretty easy to just delete that file. This program deleted about 95% of my pending list. Which is to say that 95% of spam comes from invalid addresses. Only 5% of email that I challenge comes from real working addresses. I would *LOVE* if I could prevent the 5% of people who are wrongly getting my challenges from getting them. But, by definition, I don't know who they are. As a consequence I can't possibly know whether they're sending spam or not.
The solution for that 5% of people is to use TMDA. Then they will know, just as I do, which email responses are from email that they legitimately sent and which ones are a result of forged addresses. And then they can just reject bounces from the forged stuff just like I do.
If you want to stop spammers you have to stop them from stealing bandwidth. To date, the ONLY effective solution thus far has been relay blacklisting.
And yet spam still gets thourgh RBLs. The question isn't whether or not this happens, but what to do when it happens. When spam gets through an RBL, that's when you start employing additional features. You've already lost the resources. At this point, is it worth any additional computational resources to deal with it? If the answer is no, then you have to deal with it by pressing the delete button. IMHO, I'd rather dedicated automated resources for dealing with spam rather than me.
IMHO, the problem with SPAM isn't the theft of automated resources (e.g. bandwidth, cpu, etc). This *can* become a big problem, but it's not the problem that I get frustrated with. The problem with spam is when it requires that I engage brain capacity. When it steals MY time and MY resources is when I want to delegate this effort to a computer.
Of course if my MTA signed my messages with a random key, and the challenge message sent the key back, my MTA could filter out anything I didn't actually send. Unfortunately that requires coordination which the various email/spam task groups do not seem to be capable of.
At least one C/R system does this. It does this by being able to determine legitimate email that you sent from illegitimate email. The way it does this is it tags the From address of email that you send with a cryptographic key. All responses (challenges/bounces/etc) to email that I sent will be delivered back to an address with a crypto key in it. Thus I can tell which email I've sent and which email was sent by someone else forging my address.
In my case, I interract with a lot of TMDA users. The ONLY challenges and bounces that I see are from email that I sent. All challenges or bounces sent to my mailbox as a result of someone forging my email address get dumped into the SPAM bucket. TMDA knows that I didn't send that email so it knows exactly what to do with it.
There have been dozens of these wildly espoused challenge/response systems over the years. They don't work because users hate them, because vital automated systems such as bill payment and delivery verifications can't get past them.
I've been using Challenge/Response for nearly 3 years. And I disagree with your critiques. Let's take this point by point:
Users hate them: There is a kernel of truth to this. Some users do hate them. Those users hate challenge/response so much that they instigate fights. They submit their IP addresses to RBLs for blacklisting. These are a very annoying, and vocal MINORITY. By far most users are agnostic. They deal with the challenge once and then they're done.
automated systems can't get past them: Again, there's a kernel of truth here. If you have badly configured your C/R you're going to be in trouble. But a properly configured C/R has absolutely no problems.
I use TMDA. I've got it configured so that any email I send to unknown addresses will be allowed to respond for 7 days. After that, they go into C/R. For my bill pay services, I give them a special address that allows them in forever, but that's tied to them so that I'll know if they ever hand it out to someone else.
they're almost always subverted: Really? In the last month I've had over 4000 pieces of email delivered to me from unknown addresses. Only 10 of those have been confirmed. Of the ones that were confirmed 2 of them were spam. This was easily remidied by removing those 2 addresses from my whitelist and adding them to my blacklist.
never will gain the acceptance of the user community enough to become effective: While C/R may never gain the acceptance of the user community, I don't think it's for the reasons that you cited. I think the reason is that it's too hard to set up correctly. But that being said, it doesn't need the acceptance of the user community to be effective. It works for me today whether or not you use it.
Personally, I think it'd be better if the entire world started using C/R. It'd be better because then everyone would understand that sending email to an unknown party involves a formal introduction process. This would cut down on the number of people who get confused when they receive a challenge. But if this doesn't happen it's not that big a deal. The number of confused people is already small.
IMHO, what you don't know about C/R is quite large.
Uh.. that was *NOT* a bad call. It was an illegal procedure. The call was correct. Just because it happened to take away a touchdown does not mean it was an incorrect call.
My $.02: I hate MS's products, too, but...
on
Keeping Microsoft Happy
·
· Score: 2, Insightful
...why is this op/ed piece in a section titled "News"?
Here are some excerpts from the piece:
But it's about time we started asking hard questions about where our competitiveness is taking us and who is pushing the agenda. How is it that with one of the richest corporations in the world in our backyard, our state has become less livable?
...
These aren't improvements with which Microsoft wishes to help. These are areas of concern the company wants remedied at taxpayer expense.
...
Ballmer had to know, however, that Microsoft wouldn't be footing much of the bill if taxpayers increased education funding.
...
Microsoft has been hypocritical about taxes and education.
How can anyone call those things "facts"? Their opinion. Now, I don't mind op/ed pieces. But this is reported under the title of "News". If you want to express your opinions, that's fine. Just don't tell me you're trying to express fact when you're expressing opinions.
If we in the OSS world, want to beat Microsoft, we can't accuse them of FUD at the same time that we're practicing it.
Slashdot Mirror
I truly wish there was a way for me to prevent innocents from getting unnecessary challenges. And my response to that is this: Please use SPF. If you're using SPF, I won't challenge you when someone joe-jobs your email. As soon as someone comes out with a DomainKeys filter, I'll use that, too. I'll filter on anything that can deterministically identify a forgery. But that's not today's biggest problem. By far, today, most SPAM comes from non-existant email addresses. 95% by my sampling. Which means that most of the challenges produce the desired result: preventing the spam from being delivered. A small amount (5%) get delivered. Of the ones that get delivered, I would agree that most of them go to joe-jobbed accounts.
All I can say about that is "Sorry". I will do whatever I can (short of turning of C/R) to help alleviate that. So if you have some suggestions, please provide them. My suggestions are as follows:
But you're wrong on a number of your facts. I don't think it's an inaccurate conclusion to state that you don't understand something when many of your facts turn up wrong.
There's a published standard which TMDA complies with to announce that it's challenges are machine generated. You can filter on it or not, but it's the way that all machine generated emails are supposed to be advertised.
And for what it's worth,
- manually mainting white/list black lists is very infrequent and optional
- no one has to create new accounts for every service that sends email automatically. You simply create a new email address that goes to the same account. This can be done through a webpage.
- You certainly can whitelist your friends manually, but in practice this is an optional activity.
$.02This is not unlike using keyword addresses in TMDA. The difference is that you generate the addresses and you don't have to keep track of them anywhere. The address has a hash embedded in it that's generated with a private key. When TMDA sees a keyword address it attempts to regen the hash and if it matches, the email is allowed in.
Same effect as a gold list without the list.
You might think that this is a major flaw, but most C/R solutions have already anticipated this and come up with a solution. Here's the solution using TMDA.
Yes it is, and I whole heartedly recommend to anyone doing C/R that they first filter incoming email through SPF. Thus they'll be able to identify at least some of the forgeries. If everyone were using SPF, then forgeries would not be a problem.
There are other C/R solutions that encode the information in the Subject. But it's possible to change the subject in the reply and remove the information. This is why TMDA encodes the information in the email address.
Ah. I see what you're saying. You're trying to use TMDA for a host of users. Well, frankly, I'm unqualified to answer how well that works as I haven't tried it. I use it only for myself, my wife and one of my children old enough to handle email.
At first glance I could see how it might be difficult to manage multiple accounts. That being said, there are free and commercial email services offerinng TMDA to the public. They seem to have figured out how to do so on a scalable fashion. I would recommend you talk to them about how to do it.
In one sense, you could argue that I caused the break into your house. The fact that I locked my door caused him not to break into my house and consequently he broke into your house. But the criminal here is not me for locking my door. Nor is it you for not locking your door. It's the criminal for breaking in.
My suggestion to you is to lock your door. No offense, but I'm going to reject your suggestion that I unlock mine.
You are entitled to your opinion. I disagree. Just FYI, careful how you defend yourself or your family. Because if I'm a spammer, it takes no effort for me to concocat a scenario which (by your standard) makes you a murderer for defending yourself.
While I wish I were Jason Masteler, I, unfortunately am not.
It is true, however, that you do need your own domain. But the address manipulation is not at the domain level. It's at the username level. E.g. joe@blow.com becomes joe-dated-1102294815.bed09c@blow.com. You need only 1 domain, and you generate the rest from the username.
I don't really think you know how TMDA works. I would suggest you investigate it before you make claims about it's scalability.
Just a thought. Ignore it if it's not useful to you.
Here's another scenario: using agressive spam filters, your "oppurtunity" gets miscategorized as spam, and I never even know that you sent it to me. You conclude that I don't care for the oppurtunity, and that's the end of the story.
At least with C/R, you KNOW that my spam filter has prevented me from receiving your email. With all other spam filters, it filters silently so that NO ONE knows that it's been filtered. If it doesn't filter silently, one of us has to be notified.
If I'm notified of all email coming in, that's functionally equivalent to turning off spam filterinng. If you're notified, that's functionally equivalent to C/R. According to the anti-C/R crowd, the only acceptable thing to do is turn off spam filtering. I hope they practice what they preach.
This is easily resolved by using a keyword address
I don't see how this is my (or any other C/R user's) loss. You're the one who sent the email in the first place. Presumably you had some reason for doing so. Whatever that reason was, that's what is lost. If your email wasn't important enough to you to ensure it gets to me, then I don't feel much loss for having ignored it.
And you're a very strange friend, who values a particular email system over your relationship with that person. Good luck with that.
By properly configuring the C/R system.
I'm not removing invalid email addresses before I send email to them. I'm removing it after I discover that they're invalid. How exactly does anyone know if an email address is valid before probing to see if it's valid?
Wrong again. The spam originated from someone else. My system didn't stop the spam, but you can't call my replying to an email address as a spam. That's like saying that you're a murderer if you defend yourself when a knife bounces off of your shield into someone else. The person responsible is the person who threw the knife, not the person who defended themself.
I'm sorry that someone who joe-jobs your email address causes you problems. And I'm sorry that I don't have enough information to prevent them from abusing my system to annoy you. But remember who is causing the problem: the joe-jobber. The joe-jobber is responsible for sending the spam to you in EXACTLY the same way that he's responsible for causing your email address to get bounces from bad joe-jobbed email.
My best recommendation to you:
Or don't. I don't care how you choose to defend your mailbox. Just stop calling other people "spammer" when you choose to leave your box unprotected.
- It sets the "Precedence: bulk" header.
- It sets the "Auto-Submitted: auto-replied" header.
- In the body of the message, as the first line, it says: "This message was created automatically by mail delivery software (TMDA)."
- The confirmation is actually a MIME attachment with the following MIME header: "Content-Description: Confirmation Request".
If you don't want to receive TMDA challenges, that's no problem. You can EASILY filter for any of these things and you'll never receive a TMDA challenge ever.Wrong. You assume that all email I challenge comes from legitimate addresses that are forged. While you're right that the number of spams that come in from forged addresses is greater than zero, it's no where near 100%.
I used to have a program installed that would delete emails from my pending list that were from invalid adddresses. An invalid address was determined by getting a bounce when it failed delivery. The bounce would contain a reference to the file in my pending list and it was pretty easy to just delete that file. This program deleted about 95% of my pending list. Which is to say that 95% of spam comes from invalid addresses. Only 5% of email that I challenge comes from real working addresses. I would *LOVE* if I could prevent the 5% of people who are wrongly getting my challenges from getting them. But, by definition, I don't know who they are. As a consequence I can't possibly know whether they're sending spam or not.
The solution for that 5% of people is to use TMDA. Then they will know, just as I do, which email responses are from email that they legitimately sent and which ones are a result of forged addresses. And then they can just reject bounces from the forged stuff just like I do.
And yet spam still gets thourgh RBLs. The question isn't whether or not this happens, but what to do when it happens. When spam gets through an RBL, that's when you start employing additional features. You've already lost the resources. At this point, is it worth any additional computational resources to deal with it? If the answer is no, then you have to deal with it by pressing the delete button. IMHO, I'd rather dedicated automated resources for dealing with spam rather than me.
IMHO, the problem with SPAM isn't the theft of automated resources (e.g. bandwidth, cpu, etc). This *can* become a big problem, but it's not the problem that I get frustrated with. The problem with spam is when it requires that I engage brain capacity. When it steals MY time and MY resources is when I want to delegate this effort to a computer.
In my case, I interract with a lot of TMDA users. The ONLY challenges and bounces that I see are from email that I sent. All challenges or bounces sent to my mailbox as a result of someone forging my email address get dumped into the SPAM bucket. TMDA knows that I didn't send that email so it knows exactly what to do with it.
I've been using Challenge/Response for nearly 3 years. And I disagree with your critiques. Let's take this point by point:
- Users hate them: There is a kernel of truth to this. Some users do hate them. Those users hate challenge/response so much that they instigate fights. They submit their IP addresses to RBLs for blacklisting. These are a very annoying, and vocal MINORITY. By far most users are agnostic. They deal with the challenge once and then they're done.
- automated systems can't get past them: Again, there's a kernel of truth here. If you have badly configured your C/R you're going to be in trouble. But a properly configured C/R has absolutely no problems.
- they're almost always subverted: Really? In the last month I've had over 4000 pieces of email delivered to me from unknown addresses. Only 10 of those have been confirmed. Of the ones that were confirmed 2 of them were spam. This was easily remidied by removing those 2 addresses from my whitelist and adding them to my blacklist.
- never will gain the acceptance of the user community enough to become effective: While C/R may never gain the acceptance of the user community, I don't think it's for the reasons that you cited. I think the reason is that it's too hard to set up correctly. But that being said, it doesn't need the acceptance of the user community to be effective. It works for me today whether or not you use it.
IMHO, what you don't know about C/R is quite large.I use TMDA. I've got it configured so that any email I send to unknown addresses will be allowed to respond for 7 days. After that, they go into C/R. For my bill pay services, I give them a special address that allows them in forever, but that's tied to them so that I'll know if they ever hand it out to someone else.
Personally, I think it'd be better if the entire world started using C/R. It'd be better because then everyone would understand that sending email to an unknown party involves a formal introduction process. This would cut down on the number of people who get confused when they receive a challenge. But if this doesn't happen it's not that big a deal. The number of confused people is already small.
My question stated in complete awe:
Normal slashdotter question:Uh.. that was *NOT* a bad call. It was an illegal procedure. The call was correct. Just because it happened to take away a touchdown does not mean it was an incorrect call.
Here are some excerpts from the piece:
How can anyone call those things "facts"? Their opinion. Now, I don't mind op/ed pieces. But this is reported under the title of "News". If you want to express your opinions, that's fine. Just don't tell me you're trying to express fact when you're expressing opinions.
If we in the OSS world, want to beat Microsoft, we can't accuse them of FUD at the same time that we're practicing it.
$.02